Fix remote cluster seeds fallback #34090
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Recently we introduced the settings cluster.remote to take the place of search.remote for confriguring remote cluster connections. We made this change due to the fact that we have generalized the remote cluster infrastructure to also be used within cross-cluster replication and not only cross-cluster search. For backwards compatbility, when we made this change, we allowed that cluster.remote would fallback to search.remote. Alas, the initial change for this contained a bug for handlings the proxy and seeds settings. The bug for the seeds settings arose because we were manually iterating over the concrete settings only for cluster.remote seeds but not for search.remote seeds. This commit addresses this by iterating over both cluster.remote seeds and search.remote seeds. Additionally, when checking for existence of proxy settings, we have to not only check cluster.remote proxy settings, but also fallback to search.remote proxy settings. This commit addresses both issues, and adds tests for these situations.
jasontedor
added
>non-issue
review
:Search/Search
|
Pinging @elastic/es-search-aggs |
javanna
approved these changes
Sep 26, 2018
| // sort the intersection for predictable output order | ||
| final NavigableSet<String> intersection = | ||
| new TreeSet<>(Arrays.asList( | ||
| searchRemoteSeeds.keySet().stream().filter(s -> remoteSeeds.keySet().contains(s)).sorted().toArray(String[]::new))); |
There was a problem hiding this comment.
would accepting only one or the other simplify things a bit? I guess there is no reason to use the new setting and the deprecated one at the same time?
There was a problem hiding this comment.
Note that in the immediate next line, we reject duplicates. That is, we only accept one of search.remote.<cluster_alias>.seeds and cluster.remote.<cluster_alias>.seeds for a given cluster_alias.
There was a problem hiding this comment.
I noticed that but it seems that you can still use the two settings at the same time, say one for a cluster and the other for another cluster. Is that correct?
There was a problem hiding this comment.
Yes; and that strikes me as okay so that we can support existing configurations, and users can add or migrate clusters to the new settings one at a time.
…fallback * elastic/master: TEST: Add engine is closed as expected failure msg Adjust bwc version for max_seq_no_of_updates Build DocStats from SegmentInfos in ReadOnlyEngine (elastic#34079) When creating wildcard queries, use MatchNoDocsQuery when the field type doesn't exist. (elastic#34093) [DOCS] Moves graph to docs folder (elastic#33472) Mute MovAvgIT#testHoltWintersNotEnoughData Security: use default scroll keepalive (elastic#33639) Calculate changed roles on roles.yml reload (elastic#33525) Scripting: Reflect factory signatures in painless classloader (elastic#34088) XContentBuilder to handle BigInteger and BigDecimal (elastic#32888) Delegate wildcard query creation to MappedFieldType. (elastic#34062) Painless: Cleanup Cache (elastic#33963)
|
@elasticmachine run gradle build tests |
jasontedor
added a commit
that referenced
this pull request
Sep 27, 2018
Recently we introduced the settings cluster.remote to take the place of search.remote for configuring remote cluster connections. We made this change due to the fact that we have generalized the remote cluster infrastructure to also be used within cross-cluster replication and not only cross-cluster search. For backwards compatibility, when we made this change, we allowed that cluster.remote would fallback to search.remote. Alas, the initial change for this contained a bug for handling the proxy and seeds settings. The bug for the seeds settings arose because we were manually iterating over the concrete settings only for cluster.remote seeds but not for search.remote seeds. This commit addresses this by iterating over both cluster.remote seeds and search.remote seeds. Additionally, when checking for existence of proxy settings, we have to not only check cluster.remote proxy settings, but also fallback to search.remote proxy settings. This commit addresses both issues, and adds tests for these situations.
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Sep 27, 2018
* master: (25 commits) [DOCS] Synchronize location of Breaking Changes (elastic#33588) [DOCS] Synchronizes captialization in top-level titles (elastic#33605) [SQL] Clean up LogicalPlanBuilder#doJoin (elastic#34048) Fix remote cluster seeds fallback (elastic#34090) [ML][HLRC] Replace REST-based ML test cleanup with the ML client (elastic#34109) Handle MatchNoDocsQuery in span query wrappers (elastic#34106) Update MovAvgIT AwaitsFix bug url Bad regex in CORS settings should throw a nicer error (elastic#34035) [HLRC] Support for role mapper expression dsl (elastic#33745) Watcher: Reduce script cache churn by checking for mustache tags (elastic#33978) Fold EngineSearcher into Engine.Searcher (elastic#34082) Mute SpanMultiTermQueryBuilderTests#testToQuery TESTS: Enable DEBUG Logging in Flaky Test (elastic#34091) TEST: Add engine is closed as expected failure msg Adjust bwc version for max_seq_no_of_updates Build DocStats from SegmentInfos in ReadOnlyEngine (elastic#34079) When creating wildcard queries, use MatchNoDocsQuery when the field type doesn't exist. (elastic#34093) [DOCS] Moves graph to docs folder (elastic#33472) Mute MovAvgIT#testHoltWintersNotEnoughData Security: use default scroll keepalive (elastic#33639) ...
kcm
pushed a commit
that referenced
this pull request
Oct 30, 2018
Recently we introduced the settings cluster.remote to take the place of search.remote for configuring remote cluster connections. We made this change due to the fact that we have generalized the remote cluster infrastructure to also be used within cross-cluster replication and not only cross-cluster search. For backwards compatibility, when we made this change, we allowed that cluster.remote would fallback to search.remote. Alas, the initial change for this contained a bug for handling the proxy and seeds settings. The bug for the seeds settings arose because we were manually iterating over the concrete settings only for cluster.remote seeds but not for search.remote seeds. This commit addresses this by iterating over both cluster.remote seeds and search.remote seeds. Additionally, when checking for existence of proxy settings, we have to not only check cluster.remote proxy settings, but also fallback to search.remote proxy settings. This commit addresses both issues, and adds tests for these situations.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Recently we introduced the settings
cluster.remoteto take the place ofsearch.remotefor configuring remote cluster connections. We made this change due to the fact that we have generalized the remote cluster infrastructure to also be used within cross-cluster replication and not only cross-cluster search. For backwards compatibility, when we made this change, we allowed thatcluster.remotewould fallback tosearch.remote. Alas, the initial change for this contained a bug for handling theproxyandseedssettings. The bug for theseedssettings arose because we were manually iterating over the concrete settings only forcluster.remoteseeds but not forsearch.remoteseeds. This commit addresses this by iterating over bothcluster.remoteseeds andsearch.remoteseeds. Additionally, when checking for existence of proxy settings, we have to not only checkcluster.remoteproxy settings, but also fallback tosearch.remoteproxy settings. This commit addresses both issues, and adds tests for these situations.Relates #33413