Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Discover] Unskip flaky histogram test #179974

Merged
merged 1 commit into from
Apr 5, 2024
Merged

[Discover] Unskip flaky histogram test #179974

merged 1 commit into from
Apr 5, 2024

Conversation

jughosta
Copy link
Contributor

@jughosta jughosta commented Apr 3, 2024 •
edited by kibanamachine
Loading

@jughosta jughosta added release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Apr 3, 2024
@jughosta jughosta self-assigned this Apr 3, 2024
@jughosta
Copy link
Contributor Author

jughosta commented Apr 3, 2024

/ci

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jughosta

@jughosta jughosta marked this pull request as ready for review April 4, 2024 05:46
@jughosta jughosta requested a review from a team as a code owner April 4, 2024 05:46
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

kertal
kertal reviewed Apr 5, 2024
View reviewed changes
x-pack/test_serverless/functional/test_suites/common/discover/group1/_discover_histogram.ts
@@ -75,7 +74,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
const actualCount = await elasticChart.getVisualizationRenderingCount();
const expectedCount = prevRenderingCount + renderingCountInc;
log.debug(`renderings before brushing - actual: ${actualCount} expected: ${expectedCount}`);
return actualCount === expectedCount;
return actualCount <= expectedCount;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question here the skipped test is failing because

actual: 2 expected: 1

With this change it will pass when the actual count is 0,1, ... but actually when there's an extra rendering, that we try to catch, it is not caught. My question, is the behavior als flaky when trying it out locally without any change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kertal!

Currently it fails with renderings before brushing - actual: 3 expected: 4.

As @davismcphee mentioned in https://github.com/elastic/kibana/pull/178592/files we don't know why it can be 4 sometimes on serverless. Anything below 4, we can consider as a good result. Same is on line 93 https://github.com/elastic/kibana/pull/179974/files/a08221d61210bb78ffde8aa98e3d6a6a831636f3#diff-fa544caff5a4583101f0926db5dd31ce3c10983d491a97e4484126c709f715c6R93

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx @jughosta , LGTM , we could consider to skip such tests that need very specific serverless adaptation, when they are not critical like in this case

@kertal kertal self-requested a review April 5, 2024 09:39
kertal
kertal reviewed Apr 5, 2024
View reviewed changes
Copy link
Member

@kertal kertal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@kertal kertal self-requested a review April 5, 2024 09:59
kertal
kertal approved these changes Apr 5, 2024
View reviewed changes
Copy link
Member

@kertal kertal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, part 2 👍 👍

@jughosta jughosta merged commit 087a070 into elastic:main Apr 5, 2024
23 checks passed
@jughosta jughosta deleted the 173904-fix-histogram-flaky-tests branch April 5, 2024 10:01
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Apr 5, 2024
@jughosta
[Discover] Unskip flaky histogram test (elastic#179974)
dd011a9 
- Closes elastic#173904

99x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5612

(cherry picked from commit 087a070)
@kibanamachine kibanamachine mentioned this pull request Apr 5, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.13

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Apr 5, 2024
@kibanamachine @jughosta
[8.13] [Discover] Unskip flaky histogram test (#179974) (#180125)
b2285f5 
# Backport

This will backport the following commits from `main` to `8.13`:
- [[Discover] Unskip flaky histogram test
(#179974)](#179974)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Julia
Rechkunova","email":"julia.rechkunova@elastic.co"},"sourceCommit":{"committedDate":"2024-04-05T10:01:56Z","message":"[Discover]
Unskip flaky histogram test (#179974)\n\n- Closes
https://github.com/elastic/kibana/issues/173904\r\n\r\n99x\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5612","sha":"087a0705cf7879d8aca5e9482b273549c2eca05a","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:DataDiscovery","backport:prev-minor","v8.14.0"],"title":"[Discover]
Unskip flaky histogram
test","number":179974,"url":"https://github.com/elastic/kibana/pull/179974","mergeCommit":{"message":"[Discover]
Unskip flaky histogram test (#179974)\n\n- Closes
https://github.com/elastic/kibana/issues/173904\r\n\r\n99x\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5612","sha":"087a0705cf7879d8aca5e9482b273549c2eca05a"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/179974","number":179974,"mergeCommit":{"message":"[Discover]
Unskip flaky histogram test (#179974)\n\n- Closes
https://github.com/elastic/kibana/issues/173904\r\n\r\n99x\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5612","sha":"087a0705cf7879d8aca5e9482b273549c2eca05a"}}]}]
BACKPORT-->

Co-authored-by: Julia Rechkunova <julia.rechkunova@elastic.co>
@thomheymann thomheymann mentioned this pull request Apr 8, 2024
Merged
This was referenced Apr 16, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kertal kertal kertal approved these changes

Assignees

@jughosta jughosta

Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) v8.13.2 v8.14.0
Projects
None yet
Milestone
No milestone
5 participants
@jughosta @kibana-ci @elasticmachine @kibanamachine @kertal