Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SO Migration] Improve the alias detection #105758

Open
pgayvallet opened this issue Jul 15, 2021 · 6 comments
Open

[SO Migration] Improve the alias detection #105758

pgayvallet opened this issue Jul 15, 2021 · 6 comments
Labels
Feature:Migrations project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@pgayvallet
Copy link
Contributor

Currently, the system will attempt to try the migration if either the versionAlias or the currentAlias is missing.

In theory, as both alias are created at the same time atomically during the MARK_VERSION_INDEX_READY step, they should both be either existing or non-existing.

However, in some case of migration failures requiring manual intervention, a human mistake may cause to have only one or the other to be manually created.

If the versionAlias is missing, the migration system will try to attempt a migration from the current index to itself (e.g .kibana_7.14.0 to .kibana_7.14.0). If this has no effect on the data integrity, it will fail during the later steps, as we would be write-blocking what we consider to be the source index.

We should improve the checks we're doing here:

if (
// `.kibana` and the version specific aliases both exists and
// are pointing to the same index. This version's migration has already
// been completed.
aliases[stateP.currentAlias] != null &&
aliases[stateP.versionAlias] != null &&
aliases[stateP.currentAlias] === aliases[stateP.versionAlias]
) {

To either

  • fail the migration if ONLY the versionAlias is missing
  • be 'smart' and only create this versionAlias and then continue to OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT as done when both aliases are present.
@pgayvallet pgayvallet added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:Saved Objects project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient labels Jul 15, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@lukeelmers
Copy link
Member

+1 to the "be smart" approach.

If we can avoid introducing another failure scenario without increasing complexity in a significant way, it feels worth it to me.

@afharo
Copy link
Member

afharo commented Aug 16, 2021

This failed test might be related: #97942

@afharo
Copy link
Member

afharo commented Aug 24, 2021

Also this other failed test might be related: #98463

@afharo
Copy link
Member

afharo commented Jun 27, 2022

Bumping this issue for discussion. I'm reviewing some failed tests and they seem to be related to this issue. @elastic/kibana-core should we increase the priority on this one?

@rudolf
Copy link
Contributor

rudolf commented Jun 30, 2022

@afharo I believe these failed errors come from esArchiver, left a comment #121050 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Migrations project:ResilientSavedObjectMigrations Reduce Kibana upgrade failures by making saved object migrations more resilient Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

No branches or pull requests

5 participants