Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wrong status code for SearchPhaseExecutionException #19851

Closed

Conversation

qwerty4030
Copy link
Contributor

Fix wrong status code for SearchPhaseExecutionException.
Updated SearchPhaseExecutionException to determine the status from the cause if one exists and there were no shard failures.

Throw exception if (date) histogram order path is invalid and there is only one bucket.
Forced check of sub-aggregation names and fields used in (date) histogram order path if there is only one bucket. The previous code relied on the sorting code (bypassed if less than 2 buckets) to do this check. Ideally these checks should be performed during parsing instead of the reduce phase.

Tests pass: gradle test and gradle core:integTest

Closes #14771

Updated SearchPhaseExecutionException to determine the status from the cause if one exists and there were no shard failures.

Throw exception if (date) histogram order path is invalid and there is only one bucket.
Forced check of sub-aggregation names and fields used in (date) histogram order path if there is only one bucket. The previous code relied on the sorting code (bypassed if less than 2 buckets) to do this check. Ideally these checks should be performed during parsing instead of the reduce phase.

Closes elastic#14771
@@ -396,6 +396,12 @@ public InternalAggregation doReduce(List<InternalAggregation> aggregations, Redu
} else {
// sorted by sub-aggregation, need to fall back to a costly n*log(n) sort
CollectionUtil.introSort(reducedBuckets, order.comparator());
if (reducedBuckets.size() == 1) {
// hack: force check of sub-aggregation names and fields if there is only 1 bucket (sort code bypassed)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently if the (date) histogram is ordered by sub-aggregation(s), the order path is validated during the reduce phase. This check occurs implicitly in the comparator when the histogram buckets are sorted. However if there are 0 or 1 buckets, there is nothing to sort so this code was bypassed. I added a hack here to catch the case with 1 bucket. To catch the case with 0 buckets, the validation code needs to be refactored to run during the query parsing phase (if possible). This would require parsing all sub-aggregations first and then validating the order path.

@clintongormley
Copy link

@colings86 could you review this one please?

@colings86
Copy link
Contributor

@qwerty4030 Thanks for raising this PR. I would personally prefer to fix this at parse time rather than trying to change the logic of the SearchPhaseExecutionException#status(). This is for a couple of reasons:

  • I am a bit uncomfortable about having an exception where the status code it reports changes based on the cause since the cause can often be making different assumptions of who the user is (e.g. developer v.s. user). We could end up with a library throwing e.g. an IllegalArgumentException because of something other than data the user provides (e.g. bad data in a document, internal node state etc.) and we would incorrectly return a 400 status even though the problem is not actually with the request.
  • As you also point out, the PR fixes the case where there is an invalid order when only one bucket is returned by using a hack and comparing that bucket with itself but is not able to fix the case where no buckets are returned.

Because of this I would rather fix this at parsing time, meaning that we can throw a SearchParseException which is in line with the other validation code for the search API and also meaning we can cover all cases and catch it early.

@qwerty4030
Copy link
Contributor Author

@colings86 Thanks for taking a look. I agree; catching this during parsing is better than adding a hack just for this one case. I took a quick look at the code and had some ideas, will comment in #20003.

@qwerty4030 qwerty4030 closed this Aug 20, 2016
@qwerty4030 qwerty4030 deleted the fix/14771_histogram_bad_order branch August 27, 2016 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants