Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an Elasticsearch reindex helper function #269

Open
newgene opened this issue Nov 15, 2022 · 0 comments
Open

Create an Elasticsearch reindex helper function #269

newgene opened this issue Nov 15, 2022 · 0 comments
Assignees

Comments

@newgene
Copy link
Member

newgene commented Nov 15, 2022

This should be a standalone helper function (e.g. can be under utils/es_reindex.py) used only from the Python/iPython console manually when needed. It helps to reindex an existing index by transferring both the settings, mappings and docs.

Elasticsearch's reindex API should be used, however mappings and settings from the old index should be used to create a new empty target index first. Then reindex API can be called to transfer all docs to the new index. Optionally, the alias should be switched over to the new index too. This is useful when we need to migrate existing indices created from the older ES version to the current ES version.

def reindex(src_index, target_index=None, settings=None, mappings=None, alias=None, delete_src=False):

  • target_index: use <src_index_name>_reindexed as default if None
  • settings: if provided as a dict, update the settings with the provided dictionary. Otherwise, keep the same from the src_index
  • mapping: if provided as a dict, update the settings with the provided mappings. Otherwise, keep the same from the src_index
  • alias: if True, switch the alias from src_index to target_index. If src_index has no alias, apply the <src_index_name> as the alias; if a string value, apply it as the alias instead
  • delete_src: If True, delete the src_index after everything is done

And after reindex, please also do a refresh & flush and then double-check the doc counts to make sure they are equal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants