Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and re-enable legacy import functional tests #114053

Closed
tylersmalley opened this issue Oct 6, 2021 · 10 comments
Closed

Fix and re-enable legacy import functional tests #114053

tylersmalley opened this issue Oct 6, 2021 · 10 comments
Labels
bug Fixes for quality problems that affect the customer experience failed-test A test failure on a tracked branch, potentially flaky-test impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc test-failure-flaky

Comments

@tylersmalley
Copy link
Contributor

tylersmalley commented Oct 6, 2021

These tests failed on the backport of #113680, and ended up needing to be skipped.

Failures:

  • [job] [logs] OSS CI Group #4 / dashboard app using legacy data bwc import "before all" hook in "bwc import"
  • [job] [logs] OSS CI Group #8 / management import objects .json file should provide dialog to allow the importing of saved objects with index pattern conflicts
  • [job] [logs] OSS CI Group #7 / management mgmt saved objects should import saved objects mgmt

I went through the tests manually and was not able to reproduce them, so seems to be a race condition of sorts with the test.

@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 6, 2021
@tylersmalley tylersmalley added bug Fixes for quality problems that affect the customer experience Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc labels Oct 7, 2021
@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 7, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@tylersmalley tylersmalley changed the title Fix and re-enable legacy import Fix and re-enable legacy import functional tests Oct 7, 2021
@pgayvallet
Copy link
Contributor

@tylersmalley

will expand once #113680 is merged

I think it is now!

@exalate-issue-sync exalate-issue-sync bot added impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort labels Nov 4, 2021
@tylersmalley
Copy link
Contributor Author

@pgayvallet, sorry for the delay, I have updated the description.

@lukeelmers lukeelmers added failed-test A test failure on a tracked branch, potentially flaky-test test-failure-flaky labels Nov 19, 2021
@TinaHeiligers
Copy link
Contributor

TinaHeiligers commented Dec 15, 2021

Legacy imports aren't supported in 8.0 and we deleted the tests:

Since legacy format imports are theoretically still supported in 7.16, there isn't an "easy" option to delete them from those branches. We'll need to get them re-enabled.

@exalate-issue-sync exalate-issue-sync bot added loe:large Large Level of Effort loe:medium Medium Level of Effort and removed loe:small Small Level of Effort loe:large Large Level of Effort labels Dec 16, 2021
@TinaHeiligers
Copy link
Contributor

TinaHeiligers commented Dec 16, 2021

@tylersmalley these tests are still failing on CI but pass locally 😦 . I see your initial attempt at re-enabling them failed the build too.
Is there anything we can do to help here?

@tylersmalley
Copy link
Contributor Author

@TinaHeiligers, all the failures are regarding not being able to locate the index pattern as seen in the screenshots. I, too was unable to reproduce locally. I am prett sure it's timing related, so I will leave it up to your team how to proceed.

image

@gsoldevila
Copy link
Contributor

gsoldevila commented Mar 15, 2022

I isolated one of the failed tests (bwc import) and added a debugging statement to shed some light.

The debugging exercise confirms that there is a discrepancy in behavior between the local environment and the CI environment:

During the import file, the UI calls the resolveSavedObjects method to start processing the JSON data from dashboard_6_0_1.json (legacy import that is no longer available on 8.0).

The aforementioned method returns different values:

Local environment
"🚨 resolveSavedObjects(...)" "conflictedIndexPatterns: 1; failedImports: 0"

CI tests (link here)
[00:00:31] │ debg browser[INFO] http://localhost:5620/46705/bundles/plugin/savedObjectsManagement/kibana/savedObjectsManagement.plugin.js 0:5814 "🚨 resolveSavedObjects(...)" "conflictedIndexPatterns: 0; failedImports: 1"

This, in turn, triggers different UI flows in the flyout.tsx component, and causes the functional test to fail.

I believe the other failing tests reported in this issue are due to the same problem, since the errors captured in the screenshots are the same.

Now we need to find out what is causing this discrepancy.

@gsoldevila
Copy link
Contributor

Drilling down a bit further, it seems that the errors that occur during the JSON file processing have slightly different names for their constructors:

Local environment: error.constructor.name: SavedObjectNotFound, error.savedObjectType: index-pattern.
CI: error.constructor.name: errors_SavedObjectNotFound, error.savedObjectType: index-pattern.

That's what's causing the errors to be treated differently.

@pgayvallet
Copy link
Contributor

pgayvallet commented Mar 16, 2022

it seems that the errors that occur during the JSON file processing have slightly different names for their constructors:

Very interesting, and would totally make sense to have this broken during a Babel update.

Now the main question, why would this be different between CI and Local?

@tylersmalley Are you aware of any difference on the way Babel perform its transpilation between a local and CI run?

@gsoldevila
Copy link
Contributor

gsoldevila commented Mar 17, 2022

Whilst waiting for feedback from @elastic/kibana-operations regarding the reason behind the errors_ prefix, I prepared a fix that does not rely on error.constructor.name. Let me know WDYT.

We still need to clarify the source of the discrepancy though, as it might be impacting other places where error.constructor.name is being used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience failed-test A test failure on a tracked branch, potentially flaky-test impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc test-failure-flaky
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants