Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve user-experience when encountering unknown or corrupted SO documents during a migration #129018

Closed
Tracked by #129016
pgayvallet opened this issue Mar 31, 2022 · 1 comment
Labels
Feature:Migrations impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. loe:medium Medium Level of Effort project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@pgayvallet
Copy link
Contributor

pgayvallet commented Mar 31, 2022

Part of #126672
Part of #129016

At the moment, when unknown or corrupted saved object documents are encountered during a migration, it forces the user to manually interact with the Kibana system indices to either fix or remove them before reattempting the migration.

In the scope of #126672, we'll want to:

  • Add a new migration.ignoreCorruptObjects (name TBD) that would just ignore the faulty documents during migration when enabled (disabled by default)
  • Allowlist this new config option for cloud customers

Also, in the scope of #129016, we should:

  • Assign a specific error code to this failure
  • Add online documentation describing how to fix, or work around, the failure
    • how to manually delete/fix the doc AND/OR
    • how to use the new migration config option we'll be implementing in the scope of this issue
  • Surface the error code, and the link to the documentation, in the failure's log

Open questions:

  • should we archive the faulty documents somewhere when the migration.ignoreCorruptObjects option is enabled?
    • Given we will not remove them from the old version's index, it feel acceptable to not do that
  • do we need to output more data when listing the corrupted documents in the logs?
    • Given we'll link to documentation that will explain how to retrieve the documents to fix them, I'd say this is probably unnecessary too
@pgayvallet pgayvallet added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient Feature:Migrations labels Mar 31, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Migrations impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. loe:medium Medium Level of Effort project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

No branches or pull requests

2 participants