Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass cacert file with the correct type to the elasticsearch loader #6678

Merged
merged 3 commits into from
Apr 5, 2018

Conversation

ph
Copy link
Contributor

@ph ph commented Mar 27, 2018

We were sending the cacert as [/tmp/cacert] instead of
[]string{"/tmp/cacert"} making the Elasticsearch loader crash with a
file not found.

@ph ph added the libbeat label Mar 27, 2018
@ph
Copy link
Contributor Author

ph commented Mar 27, 2018

With the fix:

 ✘  ph@sashimi  ~/go/src/github.com/elastic/beats/libbeat/dashboards   fix/support-tls-import-dashboard ± ./import_dashboards -cacert /Users/ph/es/elastic-tls-testing/certs/ca/ca.crt -file ./testdata/testbeat-dashboards.zip -es https://localhost:9200 -user elastic -pass changeme
Initialize the Elasticsearch 6.2.3 loader
Elasticsearch URL https://localhost:9200
For Elasticsearch version >= 6.0.0, the Kibana dashboards need to be imported via the Kibana API.

Without the fix:

fail to create the Elasticsearch loader: Error creating Elasticsearch client: 1 error: open [/Users/ph/es/elastic-tls-testing/certs/ca/ca.crt]: no such file or directory reading [/Users/ph/es/elastic-tls-testing/certs/ca/ca.crt]
Exiting

@ph
Copy link
Contributor Author

ph commented Mar 27, 2018

We seems to have quite a few things that break on this branch, the error related to elasticsearch probably mean the snapshot doesn't exist. I will take a quick look at this.

Copy link
Contributor

@tsg tsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@kvch
Copy link
Contributor

kvch commented Mar 28, 2018

@ph are the errors related to your change?

@andrewkroh
Copy link
Member

With this fixed, the one thing I notice is that when import_dashboards fails to connect to ES due to certificate issues it doesn't tell you that it's a certificate problem. It just says that it couldn't connect. Do you see this too if you use an invalid CA cert?

@ph
Copy link
Contributor Author

ph commented Mar 28, 2018

@kvch Not at all related, I having a hard time to run the 5.6 suite locally without my patch.

@andrewkroh Same errors as your, let me check If I can bubble up something better without making too much changes.

@ph
Copy link
Contributor Author

ph commented Mar 28, 2018

@andrewvc It's a bit of the downside of using the same package that we use inside the elasticsearch output. The binary import_dashboards doesn't expose the logs. If logs were enabled we could see the real errors which is meaningful in that case.

I could change the code to use the multierror package and return all the errors, I don't believe it would be dangerous in that case because we return an adhoc error. It would be surprising that consumer of that library assert on the content of the error.

WDYT?

Get https://localhost:9200: x509: certificate signed by unknown authority

@ph
Copy link
Contributor Author

ph commented Mar 28, 2018

@andrewkroh that would give us something like

fail to create the Elasticsearch loader: Error creating Elasticsearch client: couldn't connect to any of the configured Elasticsearch hosts, errors: 1 error: Get https://localhost:9200: x509: certificate signed by unknown authority
  • This might expose unwanted things in the logs.

@ph ph added the review label Mar 30, 2018
@ph ph force-pushed the fix/support-tls-import-dashboard branch from f04063b to fd6e4a2 Compare March 30, 2018 18:01
@ph
Copy link
Contributor Author

ph commented Apr 4, 2018

@andrewkroh @kvch @tsg

I've fixed some of the failling tests in this PR which are related to 5.6.X and not on feature created in this PR, I would be in favor to get that PR merged in and create another issue to fix the 5.6.x branch.

@ph
Copy link
Contributor Author

ph commented Apr 4, 2018

The faillure on Travis/Darwin might be related to golang/go#19734 since we are still running 1.7.6

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but can you add a CHANGELOG entry please.

ph added 3 commits April 5, 2018 09:29
We were sending the cacert as `[/tmp/cacert]` instead of
[]string{"/tmp/cacert"} making the Elasticsearch loader crash with a
file not found.
@ph ph force-pushed the fix/support-tls-import-dashboard branch from e74a1ac to c4b83fe Compare April 5, 2018 13:29
@ph
Copy link
Contributor Author

ph commented Apr 5, 2018

@andrewkroh I have added the changelog and squashed the history into meaningful commits.

@andrewkroh andrewkroh merged commit 4d1905a into elastic:5.6 Apr 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants