Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Zen2] Implement basic cluster formation #33668

Merged
merged 55 commits into from
Sep 17, 2018
Merged

[Zen2] Implement basic cluster formation #33668

merged 55 commits into from
Sep 17, 2018

Conversation

DaveCTurner
Copy link
Contributor

@DaveCTurner DaveCTurner commented Sep 13, 2018 •
edited
Loading

This PR integrates the following pieces of machinery in the Coordinator:

  • discovery
  • pre-voting
  • randomised election scheduling
  • joining (of a new master)
  • publication of cluster state updates

Together, these things are everything needed to form a cluster. We therefore
also add the start of a test suite that allows us to assert higher-level
properties of the interactions between all these pieces of machinery, with as
little fake behaviour as possible. We assert one such property: "a cluster
successfully forms".

@DaveCTurner DaveCTurner added >enhancement WIP v7.0.0 :Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. labels Sep 13, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

ywelsch
ywelsch reviewed Sep 13, 2018
View reviewed changes
Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just looked at production code so far, follow-up review for tests will follow. Looks great already.

server/src/main/java/org/elasticsearch/cluster/coordination/CoordinationState.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/Coordinator.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/JoinHelper.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/cluster/coordination/PreVoteCollector.java Outdated
super(settings);
state = new Tuple<>(null, preVoteResponse);
state = new Tuple<>(null, new PreVoteResponse(0, 0, 0));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what was the reason for this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are required to keep the PreVoteCollector abreast of changes in the current pre-vote response via the update() method, but we create it before we know what the initial response should be so have to pass in this dummy value to the constructor and call update() after loading the persistent state. On reflection we don't need to set state at all - see 98e7ad6.

server/src/main/java/org/elasticsearch/discovery/PeerFinder.java Outdated Show resolved Hide resolved
ywelsch
ywelsch approved these changes Sep 14, 2018
View reviewed changes
Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Good stuff!

@DaveCTurner DaveCTurner changed the title Start work on coordinator tests [Zen2] Integrate the components required to fully form a cluster Sep 14, 2018
@DaveCTurner DaveCTurner changed the title [Zen2] Integrate the components required to fully form a cluster [Zen2] Implement basic cluster formation Sep 14, 2018
@DaveCTurner DaveCTurner removed the WIP label Sep 14, 2018
@ywelsch ywelsch mentioned this pull request Sep 14, 2018
Closed
61 tasks
@DaveCTurner
Copy link
Contributor Author

@elasticmachine test this please

@DaveCTurner
Copy link
Contributor Author

sigh. @elasticmachine retest this please

@elasticdog
Copy link
Contributor

This job triggered CI during a migration of the master. Kicking off an additional build for you manually...

Jenkins, test this please.

@DaveCTurner
Copy link
Contributor Author

This has passed all the relevant bits of CI and failed, again, in something unrelated, so we're merging it.

@DaveCTurner DaveCTurner merged commit c79fbea into elastic:zen2 Sep 17, 2018
@DaveCTurner DaveCTurner deleted the 2018-09-11-coordinator-tests branch September 17, 2018 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ywelsch ywelsch ywelsch approved these changes

Assignees
No one assigned
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. >enhancement v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@DaveCTurner @elasticmachine @elasticdog @ywelsch @colings86