Fix TransportFieldCapabilitiesAction Blocking Transport Thread #75022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Running a request per index could take a very long time here if the request covers a larger number of shards (especially when security is enabled). Forking it to the management pool saves the transport thread from getting blocked. Also, to make this request run quicker (again esepecially with security enabled) I removed the redundant index-level request fan-out here as well to save one step of redundant request handling and authorization (the shard level request is authorized separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can refactor away the redundant index-level fan out as well.
original-brownbear
added
>non-issue
:Data Management/Indices APIs
|
Pinging @elastic/es-core-features (Team:Core/Features) |
|
@elasticmachine update branch |
jimczi
approved these changes
Jul 7, 2021
|
Thanks Jim! |
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Jul 7, 2021
…ic#75022) Running a request per index could take a very long time here if the request covers a larger number of shards (especially when security is enabled). Forking it to the management pool saves the transport thread from getting blocked. Also, to make this request run quicker (again especially with security enabled) I removed the redundant index-level request fan-out here as well to save one step of redundant request handling and authorization (the shard level request is authorized separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can refactor away the redundant index-level fan out as well.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Jul 7, 2021
We have a number of failures in these tests due to elastic#75022 which makes the field caps transport action fork to the management pool. -> we have to always fork in this test now to not block the management pool.
original-brownbear
added a commit
that referenced
this pull request
Jul 7, 2021
We have a number of failures in these tests due to #75022 which makes the field caps transport action fork to the management pool. -> we have to always fork in this test now to not block the management pool.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Jul 8, 2021
We have a number of failures in these tests due to elastic#75022 which makes the field caps transport action fork to the management pool. -> we have to always fork in this test now to not block the management pool.
original-brownbear
added a commit
that referenced
this pull request
Jul 8, 2021
… (#75081) Running a request per index could take a very long time here if the request covers a larger number of shards (especially when security is enabled). Forking it to the management pool saves the transport thread from getting blocked. Also, to make this request run quicker (again especially with security enabled) I removed the redundant index-level request fan-out here as well to save one step of redundant request handling and authorization (the shard level request is authorized separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can refactor away the redundant index-level fan out as well.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Jul 8, 2021
Follow up to elastic#75022 removing now outdated BwC code from `8.x` + some additional related dead BwC removal. In `7.x` we can't drop the index-level transport action I think because of technical BwC with the transport client so this is just an 8.x PR.
original-brownbear
added a commit
that referenced
this pull request
Jul 8, 2021
Follow up to #75022 removing now outdated BwC code from `8.x` + some additional related dead BwC removal. In `7.x` we can't drop the index-level transport action I think because of technical BwC with the transport client so this is just an 8.x PR.
jtibshirani
added
the
:Search Foundations/Mapping
|
Pinging @elastic/es-search (Team:Search) |
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Aug 11, 2021
…ic#75022) (elastic#75081) Running a request per index could take a very long time here if the request covers a larger number of shards (especially when security is enabled). Forking it to the management pool saves the transport thread from getting blocked. Also, to make this request run quicker (again especially with security enabled) I removed the redundant index-level request fan-out here as well to save one step of redundant request handling and authorization (the shard level request is authorized separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can refactor away the redundant index-level fan out as well.
|
Any thoughts on how much this will help slow field_caps responses? |
original-brownbear
added a commit
that referenced
this pull request
Aug 12, 2021
… (#75081) (#76388) Running a request per index could take a very long time here if the request covers a larger number of shards (especially when security is enabled). Forking it to the management pool saves the transport thread from getting blocked. Also, to make this request run quicker (again especially with security enabled) I removed the redundant index-level request fan-out here as well to save one step of redundant request handling and authorization (the shard level request is authorized separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can refactor away the redundant index-level fan out as well.
|
@mattkime this won't be an order-of-magnitude improvement. I think best case this is a ~2x speedup and the real world will probably see less than 2x due to other noise. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Running a request per index could take a very long time here if the request covers
a larger number of shards (especially when security is enabled).
Forking it to the management pool saves the transport thread from getting blocked.
Also, to make this request run quicker (again especially with security enabled)
I removed the redundant index-level request fan-out here as well to save one step
of redundant request handling and authorization (the shard level request is authorized
separately again anyway). In a follow-up to 8.x because of 7.x BwC issues, we can
refactor away the redundant index-level fan out as well.