[Perf] Linux/x64: 2 Regressions on 3/1/2023 3:09:21 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	8d938ec1d18de3211040bccca1ea73f2574dc87a
Compare	0a15c3b97dabaf94e5d4399aef2fd48c2b3f446a
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArraySetAll - Duration of single invocation	18.40 ns	28.57 ns	1.55	0.75	False
BitArrayCopyToByteArray - Duration of single invocation	234.27 ns	254.78 ns	1.09	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.565040267751193 > 20.308175139043065.
IsChangePoint: Marked as a change because one of 3/1/2023 11:57:24 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -11.323357216834808 (T) = (0 -25.411101227787523) / Math.Sqrt((5.511388371053423 / (29)) + (6.47613611691652 / (40))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (40) - 2, .025) and -0.35935746566929805 = (18.693465015308554 - 25.411101227787523) / 18.693465015308554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 254.77702236120942 > 243.88496824177082.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -34.49875373575901 (T) = (0 -258.17244261072915) / Math.Sqrt((8.09672638503565 / (30)) + (14.54964779603332 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.12000676586672787 = (230.50971697562912 - 258.17244261072915) / 230.50971697562912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

@radekdoulik looks like dotnet/runtime#80797 but not severe