Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure that stakeholders understand compounding index issues with larger data set #392

Open
kltm opened this issue Oct 2, 2024 · 0 comments

Comments

@kltm
Copy link
Member

kltm commented Oct 2, 2024

From #388 , we note that:

  • generated compressed index is 25G, instead of 8.8G
  • expanded, we also see 3x: 312G vs 101G
  • generation time is 7.2h vs 5h (so scales nicely there)

Any expansion to our Solr indexing, like more fields or stemming, could quickly compound some of the above.
As well, larger indexes will affect things like stats generation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

1 participant