Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract TransportRequestDeduplication from ShardStateAction #37870

Merged

Conversation

original-brownbear
Copy link
Member

@original-brownbear original-brownbear commented Jan 25, 2019

  • Extracted the logic for master request duplication so it can be reused by the snapshotting logic to prevent flooding master with shard-snapshot-state updates
  • Removed custom listener used by ShardStateAction to not leak these into future users of this class
  • Changed semantics slightly to get rid of redundant instantiations of the composite listener
  • Relates Fix Two Races that Lead to Stuck Snapshots #37686

* Extracted the logic for master request duplication so it can be reused by the snapshotting logic
* Removed custom listener used by `ShardStateAction` to not leak these into future users of this class
* Changed semantics slightly to get rid of redundant instantiations of the composite listener
* Relates elastic#37686
@original-brownbear original-brownbear added v7.0.0 :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. >refactoring v6.7.0 labels Jan 25, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@original-brownbear
Copy link
Member Author

@dnhatn ping (just in case this got lost, no big rush :))

@dnhatn
Copy link
Member

dnhatn commented Jan 29, 2019

@original-brownbear sorry for the delay. I will do it today.

dnhatn
dnhatn previously approved these changes Jan 30, 2019
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change. I left a some nits and a point to discuss.

/**
* Register a listener for the given request with the deduplicator.
* If the request is not yet registered with the deduplicator it will return an {@link ActionListener}
* that must be completed by the called when the request completes. If the request is already known to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the called -> the caller.

* @param listener Listener to invoke on request completion
* @return Listener that must be invoked by the caller or null when the request is already known
*/
public ActionListener<Void> register(T request, ActionListener<Void> listener) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am making a suggestion but feel free to reject it because for me this PR is very good already. How about adding a new argument Consumer<Listener> that executes the request if it was not executed before (surely we need to rename the method). With this change, I think we can safely use this method without reading the documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we miss a test which ensures that if we register a request multiple times (maybe concurrently), we return a non-null result only once. And we need also to verify that when the returned listener is completed, the cached size is reduced.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding a new argument Consumer that executes the request if it was not executed before (surely we need to rename the method). With this change, I think we can safely use this method without reading the documentation.

👍 I like it :) On it.

I think we miss a test which ensures that if we register a request multiple times (maybe concurrently), we return a non-null result only once. And we need also to verify that when the returned listener is completed, the cached size is reduced.

Sure, sounds reasonable => on it :)

@dnhatn dnhatn dismissed their stale review January 30, 2019 04:24

I think we miss a test which ensures that if we register a request multiple times (maybe concurrently), we return a non-null result only once. And we need also to verify that when the returned listener is completed, the cached size is reduced.

@original-brownbear
Copy link
Member Author

original-brownbear commented Jan 30, 2019

@dnhatn thanks for the review! All points addressed I think :)
I made a slight change of plans and went with a bi-consumer for the callback instead of a consumer so that we avoid a new object there on every invocation by passing the request as well => also made adjusting the tests as requested much nicer :)

Take a look and let me know what you think.

@original-brownbear
Copy link
Member Author

All green again :)

Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a nit on the javadocs. This is a nice refactoring. Thanks for an extra iteration @original-brownbear.

@original-brownbear
Copy link
Member Author

@dnhatn thanks for the review!

@original-brownbear original-brownbear merged commit a070b8a into elastic:master Jan 30, 2019
@original-brownbear original-brownbear deleted the extract-transport-dedup branch January 30, 2019 18:21
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Jan 30, 2019
* master:
  Remove types from watcher docs (elastic#38002)
  Add test coverage for Painless general casting of boolean and Boolean (elastic#37780)
  Fixed test bug, lastFollowTime is null if there are no follower indices.
  Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984)
  Extract TransportRequestDeduplication from ShardStateAction (elastic#37870)
  Expose retention leases in shard stats (elastic#37991)
jasontedor added a commit to dnhatn/elasticsearch that referenced this pull request Jan 31, 2019
* elastic/master:
  ILM setPriority corrections for a 0 value (elastic#38001)
  Temporarily disable BWC for retention lease stats (elastic#38049)
  Skip Shrink when numberOfShards not changed (elastic#37953)
  Add dispatching to `HandledTransportAction` (elastic#38050)
  Update httpclient for JDK 11 TLS engine (elastic#37994)
  Reduce flaxiness of ccr recovery timeouts test (elastic#38035)
  Fix ILM status to allow unknown fields (elastic#38043)
  Fix ILM Lifecycle Policy to allow unknown fields (elastic#38041)
  Update verify repository to allow unknown fields (elastic#37619)
  [ML] Datafeed deprecation checks (elastic#38026)
  Deprecate minimum_master_nodes (elastic#37868)
  Remove types from watcher docs (elastic#38002)
  Add test coverage for Painless general casting of boolean and Boolean (elastic#37780)
  Fixed test bug, lastFollowTime is null if there are no follower indices.
  Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984)
  Extract TransportRequestDeduplication from ShardStateAction (elastic#37870)
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Feb 26, 2019
…37870)

* Extracted the logic for master request duplication so it can be reused by the snapshotting logic
* Removed custom listener used by `ShardStateAction` to not leak these into future users of this class
* Changed semantics slightly to get rid of redundant instantiations of the composite listener
* Relates elastic#37686
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Feb 26, 2019
…37870)

* Extracted the logic for master request duplication so it can be reused by the snapshotting logic
* Removed custom listener used by `ShardStateAction` to not leak these into future users of this class
* Changed semantics slightly to get rid of redundant instantiations of the composite listener
* Relates elastic#37686
original-brownbear added a commit that referenced this pull request Feb 26, 2019
…39399)

* Extracted the logic for master request duplication so it can be reused by the snapshotting logic
* Removed custom listener used by `ShardStateAction` to not leak these into future users of this class
* Changed semantics slightly to get rid of redundant instantiations of the composite listener
* Relates #37686
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. >refactoring v6.7.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants