Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Zen2] Randomized testing of CoordinationState #32242

Merged
merged 3 commits into from
Aug 7, 2018

Conversation

ywelsch
Copy link
Contributor

@ywelsch ywelsch commented Jul 20, 2018

Simulates a random run of a cluster with multiple CoordinationState instances (each representing one node), passing messages back and forth, and asserting that the overall system satisfies a given set of safety properties.

Follow-up to #32171

@ywelsch ywelsch added >non-issue v7.0.0 :Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. labels Jul 20, 2018
@ywelsch ywelsch requested a review from DaveCTurner July 20, 2018 14:37
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@ywelsch ywelsch mentioned this pull request Jul 20, 2018
61 tasks
@ywelsch
Copy link
Contributor Author

ywelsch commented Aug 6, 2018

ping @DaveCTurner

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I asked for a couple of tiny changes, but definitely no need for a review.

Seems to reliably achieve at least a handful of commits during the run, which is enouraging for coverage. I've run thousands of repeats without failing an assertion, so maybe the maths works!

}
}

// check system invariants
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It caught me out that this can happen outside the loop, but on closer inspection the invariants checked are both monotonic so it's sufficient to check them at the end. I think this is worth a comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, fixed in 711447b

@@ -752,6 +759,11 @@ public void testVoteCollection() {
});
}

public void testSafety() {
Cluster cluster = new Cluster(randomIntBetween(1, 5));
cluster.runRandomly(10000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IntelliJ suggests to inline this constant, and while I'm here I suggest to inline cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 711447b

throw new CoordinationStateRejectedException("node not available");
} else {
final Object payload = message.payload;
if (payload instanceof StartJoinRequest) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay for dynamic dispatching ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)

@ywelsch ywelsch merged commit 22c3673 into elastic:zen2 Aug 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. >non-issue v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants