Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hot restart: block new starts if previous process is still initializing #553

Merged
merged 2 commits into from
Mar 10, 2017

Conversation

mattklein123
Copy link
Member

fixes #550

@mattklein123
Copy link
Member Author

@lyft/network-team

@kyessenov can you please test this PR and let me know if it fixes your issue.

@kyessenov
Copy link
Contributor

I have trouble reproducing the race reliably. We run the fast restart on each pre-submit, so I can let you know if I this happens again after merge.

@mattklein123
Copy link
Member Author

@kyessenov deploying a change like this at Lyft is kind of a pain. I would really like to know if this fixes your issue, and you are going to have to write code around it also to handle the error case. Isn't there some way you can test this locally?

@mattklein123
Copy link
Member Author

@kyessenov has confirmed fix.

@lyft/network-team please review.

@@ -53,6 +53,16 @@ SharedMemory& SharedMemory::initialize(Options& options) {
RELEASE_ASSERT(shmem->version_ == VERSION);
}

// Here we catch the case where a new Envoy starts up when the current Envoy has not yet fully
// initialized. The startup logic is quite compicated, and it's not worth trying to handle this
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo complicated

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will fix in follow up.

@mattklein123 mattklein123 merged commit 2685331 into master Mar 10, 2017
@mattklein123 mattklein123 deleted the hot_restart_init_block branch March 10, 2017 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Potential race in hot restart protocol
4 participants