Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce flakyness of Resource tests #12252

Merged
merged 6 commits into from
Sep 16, 2024

Conversation

jaydeluca
Copy link
Member

@jaydeluca jaydeluca commented Sep 14, 2024

This test is 43% flaky

image

Looking at the failures, it is that the host.arch attribute is not reliable

image

Moving the Security Manager tests to their own test suite to avoid them altering the environment, causing issues with other tests

@jaydeluca jaydeluca requested a review from a team September 14, 2024 10:19
@laurit
Copy link
Contributor

laurit commented Sep 14, 2024

I think this is not a good way to fix this issue. The reason this test is flaky (it actually reliably fails on the first execution and passes on retry) is that it depends on the order tests are run. If

is run first it will poison the environment for this test. Even if you ignore host.arch you'll next run into the same issue with os.description (at least that is what happened when I disabled HostResourceTest.SecurityManagerEnabled). Perhaps we should move all the security manager tests to a separate suite so they couldn't interfere with the main suite.
I can reliably reproduce this with running ./gradlew :instrumentation:resources:library:clean and ./gradlew :instrumentation:resources:library:test --no-build-cache -PtestJavaVersion=8

@jaydeluca jaydeluca changed the title Remove assertion for host.arch to reduce flakiness Reduce flakyness of Resource tests Sep 15, 2024
@trask trask merged commit 5e5be42 into open-telemetry:main Sep 16, 2024
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants