Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[kbn-es] await native realm setup, error if there are failures #36290

Merged
merged 14 commits into from
May 10, 2019
Merged

[kbn-es] await native realm setup, error if there are failures #36290

merged 14 commits into from
May 10, 2019

Conversation

spalger
Copy link
Contributor

@spalger spalger commented May 8, 2019

While trying to debug some transient es security failures we've been seeing I noticed that the native realm setup logic is outside of the promise chain and isn't guaranteed to be done before the promise returned by Cluster#start() resolves. Additionally, errors in that process do not bubble up, but are logged in a way that's too easy to miss.

This moves the nativeRealm setup to its own promise, along with the http port discovery, and consumes them as necessary.

@spalger spalger added review Team:Operations Team label for Operations Team v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.2.0 labels May 8, 2019
@spalger spalger requested a review from tylersmalley May 8, 2019 18:00
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations

@tylersmalley
Copy link
Contributor

Looks like the port check is failing:

(node:1847) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_URL]: Invalid URL: http://elastic:changeme@localhost:undefined
    at onParseError (internal/url.js:241:17)
    at parse (internal/url.js:257:3)
    at new URL (internal/url.js:332:5)
    at ConnectionPool.urlToHost (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/lib/ConnectionPool.js:390:12)
    at ConnectionPool.addConnection (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/lib/ConnectionPool.js:216:19)
    at new Client (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/index.js:104:27)
    at new NativeRealm (/mnt/c/elastic/kibana/packages/kbn-es/src/utils/native_realm.js:27:20)
    at _nativeRealmSetup._httpPort.then (/mnt/c/elastic/kibana/packages/kbn-es/src/cluster.js:272:27)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:1847) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:1847) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Additionally, if we are getting rid of the try/catch in isSecurityEnabled. We will need to handle that path not being present in OSS and in turn not calling setPasswords

@spalger spalger removed the request for review from tylersmalley May 8, 2019 19:35
@spalger spalger added WIP Work in progress and removed review labels May 8, 2019
@elasticmachine

This comment has been minimized.

Sign in to view
@spalger spalger requested a review from tylersmalley May 9, 2019 14:52
@spalger spalger added review and removed WIP Work in progress labels May 9, 2019
@spalger
Copy link
Contributor Author

spalger commented May 9, 2019

Fixed the port issue (typo) and filtered out 400 errors, which is what we get when xpack is not included since the _xpack path is parsed as an index name and the categories parameter is unknown triggering a Bad Request response.

@elasticmachine

This comment has been minimized.

Sign in to view
@elasticmachine

This comment has been minimized.

Sign in to view
@spalger
Copy link
Contributor Author

spalger commented May 9, 2019

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

tylersmalley
tylersmalley approved these changes May 9, 2019
View reviewed changes
Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one minor nit regarding the log message with the mocked bin.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@spalger spalger merged commit 6ef9fbe into elastic:master May 10, 2019
@spalger spalger mentioned this pull request May 10, 2019
Closed
spalger pushed a commit to spalger/kibana that referenced this pull request May 10, 2019
[kbn-es] await native realm setup, error if there are failures (elast…
8b9dd68 
…ic#36290)

* [kbn-es] await native realm setup, error if there are failures

* cleanup listener from started promise too

* fix typo

* log native realm setup errors on run and handle 400 when xpack isn't available

* avoid creating potentially unused promises

* avoid Promise constructor that's larger than necessary

* group similar operations, add some light comments

* set passwords in parallel so that logging is grouped

* update native realm tests

* update es_bin fixture to handle _xpack requests

* log started after server is listening

* remove unused mockEsBin args

* use more realistic http address log
@spalger spalger mentioned this pull request May 10, 2019
Merged
spalger pushed a commit that referenced this pull request May 10, 2019
Revert "[kbn-es] await native realm setup, error if there are failures (
a102ca1 
#36290)"

This reverts commit 6ef9fbe.
spalger pushed a commit to spalger/kibana that referenced this pull request May 10, 2019
Revert "Revert "[kbn-es] await native realm setup, error if there are…
a17c301 
… failures (elastic#36290)""

This reverts commit a102ca1.
@spalger spalger deleted the await-native-realm-setup branch May 10, 2019 19:57
spalger pushed a commit that referenced this pull request May 20, 2019
[kbn/es] await native realm setup, error if there are failures (#36475)
508424e 
* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails
spalger pushed a commit to spalger/kibana that referenced this pull request May 20, 2019
[kbn/es] await native realm setup, error if there are failures (elast…
f7973f0 
…ic#36475)

* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (elastic#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails
spalger pushed a commit that referenced this pull request May 21, 2019
[7.x] [kbn/es] await native realm setup, error if there are failures (#…
1f83652 
…36475) (#36722)

* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails
@spalger spalger mentioned this pull request May 22, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tylersmalley tylersmalley tylersmalley approved these changes

Assignees
No one assigned
Labels
release_note:skip Skip the PR/issue when compiling release notes reverted review Team:Operations Team label for Operations Team v7.2.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@spalger @elasticmachine @tylersmalley