Flaky testsuit track list #4193
Labels
CI
Continuous integration
component: cli
About the pure command line interface of the hls executable
flaky test
type: bug
Something isn't right: doesn't work as intended, documentation is missing/outdated, etc..
type: enhancement
New feature or request
Trackers
Some tests are really falky, it is important for us to take a look and solve them.It is crucial important to the HLS stability.
It should be part of #3736
GetLinkable
#4093 fixed by Stabilize the build system by correctly house keeping the dirtykeys and rule values [flaky test #4185 #4093] #4190addDependentFile
test #4194 fixed by Stabilize the build system by correctly house keeping the dirtykeys and rule values [flaky test #4185 #4093] #4190IfaceTests
, we should remove interface cache dir before we run the test, or the diganostic simply gone #4200ghcide/test/exe/UnitTests.hs:91: expected: [20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1] but got: [20,19]
TDeclKindError (golden): FAIL (66.44s) Timed out waiting to receive a message from the server. Last message received: { "jsonrpc": "2.0", "method": "$/progress", "params": { "token": "35", "value": { "kind": "end", "message": "Finished indexing 3 files" } } }
PS. I am doing some migration in #4173 collecting these
Set up long run CI to test the flaky test for 500 times
We can set up a standard to verify a flaky test is gone by running it consectively 500 times.
Since our CI is likely to miss this, we can set up a long running CI elsewhere to track our main branch and run the flaky test we pick, so we can see status of it .
@jhrcek already develop a script to run these test. we can build the long running CI based on it
The text was updated successfully, but these errors were encountered: