Allow index filtering in field capabilities API #57276
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change allows to use an `index_filter` in the
field capabilities API. Indices are filtered from
the response if the provided query rewrites to `match_none`
on every shard:
````
GET metrics-*
{
"index_filter": {
"bool": {
"must": [
"range": {
"@timestamp": {
"gt": "2019"
}
}
}
}
}
````
The filtering is done on a best-effort basis, it uses the can match phase
to rewrite queries to `match_none` instead of fully executing the request.
The first shard that can match the filter is used to create the field
capabilities response for the entire index.
Closes elastic#56195
jimczi
added
>feature
:Search/Search
|
Pinging @elastic/es-search (:Search/Search) |
jpountz
approved these changes
May 29, 2020
| static Request fieldCaps(FieldCapabilitiesRequest fieldCapabilitiesRequest) { | ||
| Request request = new Request(HttpGet.METHOD_NAME, endpoint(fieldCapabilitiesRequest.indices(), "_field_caps")); | ||
| static Request fieldCaps(FieldCapabilitiesRequest fieldCapabilitiesRequest) throws IOException { | ||
| String methodName = fieldCapabilitiesRequest.indexFilter() != null ? HttpPut.METHOD_NAME : HttpGet.METHOD_NAME; |
There was a problem hiding this comment.
PUT? shouldn't it be a POST?
server/src/main/java/org/elasticsearch/action/fieldcaps/FieldCapabilitiesIndexRequest.java
Outdated
Show resolved
Hide resolved
| @@ -67,6 +67,7 @@ public TransportFieldCapabilitiesAction(TransportService transportService, Clust | |||
|
|
|||
| @Override | |||
| protected void doExecute(Task task, FieldCapabilitiesRequest request, final ActionListener<FieldCapabilitiesResponse> listener) { | |||
| long nowInMillis = request.nowInMillis() == null ? System.currentTimeMillis() : request.nowInMillis(); | |||
There was a problem hiding this comment.
question: in what cases would the current timestamp already be set?
There was a problem hiding this comment.
If the action is executed by a cross cluster search. See https://github.com/elastic/elasticsearch/pull/57276/files?file-filters%5B%5D=.java#diff-7bbfa44263f4c8fc5d8620e78cfe088bR130.
I added a comment in fb3007c
.../src/main/java/org/elasticsearch/action/fieldcaps/TransportFieldCapabilitiesIndexAction.java
Outdated
Show resolved
Hide resolved
qa/multi-cluster-search/src/test/resources/rest-api-spec/test/remote_cluster/10_basic.yml
Show resolved
Hide resolved
jtibshirani
reviewed
Jun 1, 2020
| query rewrites to `match_none` on every shard. | ||
| + | ||
| -- | ||
| [IMPORTANT] |
There was a problem hiding this comment.
Looks like we've duplicated this whole note, maybe we could just keep the one below?
This was referenced Jun 17, 2020
jimczi
added a commit
that referenced
this pull request
Jun 18, 2020
This change allows to use an `index_filter` in the
field capabilities API. Indices are filtered from
the response if the provided query rewrites to `match_none`
on every shard:
````
GET metrics-*
{
"index_filter": {
"bool": {
"must": [
"range": {
"@timestamp": {
"gt": "2019"
}
}
}
}
}
````
The filtering is done on a best-effort basis, it uses the can match phase
to rewrite queries to `match_none` instead of fully executing the request.
The first shard that can match the filter is used to create the field
capabilities response for the entire index.
Closes #56195
jimczi
added a commit
to jimczi/elasticsearch
that referenced
this pull request
Jun 18, 2020
This commit fixes the multi cluster yaml tests by restoring the number of shards to 1 for the single_doc_index (removed in elastic#57276). Closes elastic#58345
29 tasks
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 27, 2020
This commit introduces overloads of field_caps API that do not take a body, as a BWC change for body introduced in elastic/elasticsearch#57276
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 27, 2020
This commit introduces overloads of field_caps API that do not take a body, as a BWC change for body introduced in elastic/elasticsearch#57276 (cherry picked from commit 1f02223)
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 28, 2020
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 31, 2020
github-actions bot
pushed a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 31, 2020
github-actions bot
pushed a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 31, 2020
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 31, 2020
Relates: elastic/elasticsearch#57276 Co-authored-by: Russ Cam <russ.cam@elastic.co>
russcam
added a commit
to elastic/elasticsearch-net
that referenced
this pull request
Jul 31, 2020
Relates: elastic/elasticsearch#57276 Co-authored-by: Russ Cam <russ.cam@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change allows to use an
index_filterin the field capabilities API. Indices are filtered fromthe response if the provided query rewrites to
match_noneon every shard:The filtering is done on a best-effort basis, it uses the can match phase to rewrite queries to
match_noneinstead of fully executing the request. The first shard that can match the filter is used to create the field capabilities response for the entire index.Closes #56195