Skip flakey tests in CI #50071
Skip flakey tests in CI #50071
Conversation
| @@ -189,7 +190,8 @@ end | |||
|
|
|||
| returned_mat = mul!(C, Af, Bf, α, β) | |||
| @test returned_mat === C | |||
| @test collect(returned_mat) ≈ α * Ac * Bc + β * Cc rtol=rtol | |||
| # This test is skipped because it is flakey, but should be fixed and put back (see #49966) | |||
There was a problem hiding this comment.
Ah! I thought I broke it in #49405 (comment)
There was a problem hiding this comment.
haha, I don't think so
| withenv("JULIA_NUM_THREADS" => nt) do | ||
| @test read(`$exename --threads=2 -e $code`, String) == | ||
| read(`$exename -t 2 -e $code`, String) == "2" | ||
| if Sys.iswindows() |
There was a problem hiding this comment.
I'm not entirely sure Windows is immune from issues, maybe it just happens less frequently, I seem to remember @vtjnash saying that these tests are a bit unreliable in general (do I remember wrong?). But it'd also be sad to completely miss this block of tests.
There was a problem hiding this comment.
It hasn't failed for Windows in the past 40 commits to master so I'm hopeful, but far from confident that these are reliable on Windows. Not wanting to miss this entire block also motivated my not skipping these on Windows.
There was a problem hiding this comment.
I think Windows does not have this segfault issue, but because of that, it might hang instead.
|
Do we know why windows tests have "Received cancellation signal, interrupting" and fail after 1s? |
Recent failure on AMD64: 2616634 https://buildkite.com/julialang/julia-master/builds/39998#0191ecf7-1b45-497a-997c-508be066be7f Recent failure on PowerPC: bf6962c https://buildkite.com/julialang/julia-master/builds/39932#0191e368-7016-46a2-9750-10a43bde1329 xref JuliaLang#49966 xref JuliaLang#50071
Partial revert of #50071, inspired by conversation in #49966 (comment)
Partial revert of #50071, inspired by conversation in #49966 (comment) Ran the tests 100 times to make sure we're not putting back something that's still flaky. Closes #49966
This PR marks tests that have been failing CI recently as skipped so that folks submitting unrelated PRs don't have to go through test logs to make sure that failures are unrelated (and to reduce the likelihood and frequency of this kind of mistake)
Addmul
Failure on linux 0c774c7: buildkite
Failure on windows c3d70e8: buildkite
Failure on macos 94bc2f3: buildkite
Reported: #49966
Spawn and wait lots of tasks
Failure on mac f407a4c: buildkite
Reported: #43124 (comment)
cmdlineargs with --threads
Failure on mac 9b27a8f8a5: buildkite
Failure on mac 20752db7c1: buildkite
Failure on linux d912d85c42: buildkite
Failure on linux 318f0ead95: buildkite
Failure on FreeBSD: #49072 (comment)
Reported: #49154