Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Monitoring] Improve indices loading against larger metricbeat-* indices #79190

Merged

Conversation

chrisronline
Copy link
Contributor

Resolves #78668

This is difficult to test because in the various attempts we made to figure out the issue, we added a mapping to all metricbeat-* indices that fixes it across the board and we are unable to back out of that mapping addition without reindexing.

I'll open this PR for now and think through how we can properly test this

@elasticmachine
Copy link
Contributor

Pinging @elastic/stack-monitoring (Team:Monitoring)

Copy link
Contributor

@igoristic igoristic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrisronline This looks good, but then shouldn't we do this for all our queries that are doing sort on timestamp (and maybe even other fields where we do sorting)?

I was actually thinking maybe we can have appendMetricbeatIndex return an empty string temporarily (until it actually works), wdyt?

@chrisronline
Copy link
Contributor Author

I updated the all inner_hits to use the same fix. Thanks @igoristic

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

async chunks size

id before after diff
monitoring 1.2MB 1.2MB +35.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@igoristic igoristic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome job! ...and great discovery/fix! 👏

@chrisronline chrisronline merged commit 1f56371 into elastic:master Oct 8, 2020
@chrisronline chrisronline deleted the monitoring/inner_hits_unmapped_type branch October 8, 2020 18:50
chrisronline added a commit that referenced this pull request Oct 8, 2020
…ces (#79190) (#80050)

* Improve indices loading against larger metricbeat-* indices

* PR feedback
gmmorris added a commit to gmmorris/kibana that referenced this pull request Oct 12, 2020
* master: (23 commits)
  Table visualization renderer (elastic#79455)
  Migrate Jest JUnit reporter to TS (elastic#79919)
  store sorted bundleRefExportIds (elastic#80011)
  update chromedriver dependency to 86.0.0 (elastic#79972)
  [Security Solution][Case] Fix bug when changing connectors (elastic#80002)
  [kbn/std] add observable helpers to aid with rxjs 7 upgrade (elastic#79752)
  [Security Solution][Resolver] Pill numbers in compact notation (elastic#80038)
  [Logs UI] Sync logs timerange with wider Kibana (elastic#79444)
  [DOCS] Adds quick start (elastic#78822)
  [Ingest Manager]Fix ingest manager UI renaming (elastic#80036)
  [Monitoring] Fixed internal monitoring check (elastic#79241)
  [Security Solution][Exception Modal] Removes list operators in exception modal for EQL rules (elastic#79871)
  Update development documentation about REST API best practices (elastic#80009)
  [Monitoring] Improve indices loading against larger metricbeat-* indices (elastic#79190)
  [CI] Move kibana build dir outside of repo for functional tests (elastic#80018)
  [kbn/optimizer] bump low or add missing limits when updating automatically (elastic#80013)
  [Enterprise Search] Added a Credentials page to App Search (elastic#79749)
  [DOCS] Canvas refresh for 7.10 (elastic#80017)
  [TSVB] Add ignore global filters to series options (elastic#79337)
  Remove this check (elastic#79202)
  ...
@inqueue
Copy link
Member

inqueue commented Feb 4, 2021

The issue is having major impact. Is it feasible to backport this PR to 7.10?

chrisronline added a commit to chrisronline/kibana that referenced this pull request Feb 4, 2021
…ces (elastic#79190)

* Improve indices loading against larger metricbeat-* indices

* PR feedback
@chrisronline
Copy link
Contributor Author

@inqueue Sure, #90317

chrisronline added a commit that referenced this pull request Feb 4, 2021
…ces (#79190) (#90317)

* Improve indices loading against larger metricbeat-* indices

* PR feedback
@chrisronline
Copy link
Contributor Author

@inqueue done afa4382

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Monitoring] Index listing page times out for large monitoring/metricbeat clusters
5 participants