Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monitoring UI inaccessible if remote CCS cluster is down #57596

Closed
n0othing opened this issue Feb 13, 2020 · 2 comments · Fixed by #79202
Closed

Monitoring UI inaccessible if remote CCS cluster is down #57596

n0othing opened this issue Feb 13, 2020 · 2 comments · Fixed by #79202
Labels
bug Fixes for quality problems that affect the customer experience Feature:Stack Monitoring Team:Monitoring Stack Monitoring team v7.6.0

Comments

@n0othing
Copy link
Member

Kibana version: tested on version 7.6.0

Elasticsearch version: 7.6.0

Describe the bug:

The Stack Monitoring UI becomes inaccessible if a remote CCS cluster becomes disconnected. When this happens, Monitoring documents are still being written to the .monitoring-es-7-* index.

Steps to reproduce:

  1. Start two separate clusters (e.g cluster-1 and cluster-2)
  2. Connect Kibana to cluster-1 and enable Monitoring.
  3. Connect cluster-2 to cluster-1 as a remote CCS cluster.
  4. Shut down cluster-2.
  5. Try navigating/refreshing Stack Monitoring

Screenshots (if relevant):

stack_monitoring_broken

@n0othing n0othing added bug Fixes for quality problems that affect the customer experience Feature:Stack Monitoring v7.6.0 labels Feb 13, 2020
@myasonik myasonik added the Team:Monitoring Stack Monitoring team label Feb 14, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/stack-monitoring (Team:Monitoring)

@pickypg
Copy link
Member

pickypg commented Feb 14, 2020

Workaround:

# 6.1 - 7.6
xpack.monitoring.ccs.enabled: false

# 7.7+ (looking at the repo)
monitoring.ui.ccs.enabled: false

I was going to make a docs PR, but I don't have time to deal with backporting and the setting change:

Set to false to disable the usage of Cross Cluster Search for the Monitoring UI in Kibana. This largely impacts the landing page for the Monitoring UI, which attempts to discover Monitoring data using the local and all (*) remote clusters. After selecting any monitored cluster, the UI will intelligently reuse the same cluster (remote or otherwise) to find it. Defaults to true.


Because of how this query is only sent on the landing page, the experience could be improved a little by introspecting the error returned by Elasticsearch and, if it can be detected that it relates to CCS, simply send another request without CCS and use whatever that returns (plus returning the error, which is something that would be new functionality -- data plus an error).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Stack Monitoring Team:Monitoring Stack Monitoring team v7.6.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants