[Perf] Linux/arm64: 4 Regressions on 8/4/2023 10:42:53 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d1e584d09973bc200c0ac0997d9e2c875260a1c9
Compare	d66eb5ce6257c994de2025be35c3f0db6b23e3b9
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_Monitor

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryEnterExit - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	20.92 ns	22.65 ns	1.08	0.16	False
EnterExit - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	18.23 ns	22.65 ns	1.24	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Monitor*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_Monitor.TryEnterExit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.653268112920664 > 21.866065635752054.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 7/17/2023 4:07:27 PM, 8/3/2023 11:36:41 PM, 8/9/2023 3:40:37 PM falls between 8/1/2023 12:22:20 AM and 8/9/2023 3:40:37 PM.
IsRegressionStdDev: Marked as regression because -9.202642023985499 (T) = (0 -22.79120924307673) / Math.Sqrt((0.455839246077099 / (13)) + (0.3043922285897668 / (10))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (10) - 2, .025) and -0.11525285310760162 = (20.435912071033986 - 22.79120924307673) / 20.435912071033986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_Monitor.EnterExit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.646656728600195 > 20.23092841458945.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/17/2023 4:07:27 PM, 8/3/2023 11:36:41 PM, 8/9/2023 3:40:37 PM falls between 8/1/2023 12:22:20 AM and 8/9/2023 3:40:37 PM.
IsRegressionStdDev: Marked as regression because -13.148006906432936 (T) = (0 -22.539103977245507) / Math.Sqrt((0.7822494663315064 / (13)) + (0.06421297297235101 / (10))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (10) - 2, .025) and -0.17721366550401418 = (19.146145375058637 - 22.539103977245507) / 19.146145375058637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d1e584d09973bc200c0ac0997d9e2c875260a1c9
Compare	d66eb5ce6257c994de2025be35c3f0db6b23e3b9
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Layout.SearchLoops

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LoopReturn - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	66.55 ns	70.04 ns	1.05	0.00	False
LoopGoto - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	66.57 ns	70.22 ns	1.05	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Layout.SearchLoops*'

Payloads

Baseline
Compare

Layout.SearchLoops.LoopReturn

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.04323364394249 > 69.78575293336758.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 8/3/2023 11:36:41 PM, 8/9/2023 3:40:37 PM falls between 8/1/2023 12:22:20 AM and 8/9/2023 3:40:37 PM.
IsRegressionStdDev: Marked as regression because -31.432436916705864 (T) = (0 -70.22669326533125) / Math.Sqrt((0.18419418551232128 / (13)) + (0.009944722234127132 / (10))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (10) - 2, .025) and -0.05833016963950684 = (66.35612900391206 - 70.22669326533125) / 66.35612900391206 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Layout.SearchLoops.LoopGoto

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.22145301560764 > 69.85875156332189.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 8/3/2023 2:00:30 PM, 8/9/2023 3:40:37 PM falls between 8/1/2023 12:22:20 AM and 8/9/2023 3:40:37 PM.
IsRegressionStdDev: Marked as regression because -21.19976815137259 (T) = (0 -70.19776786012002) / Math.Sqrt((0.3751588541073584 / (12)) + (0.004347295098156162 / (11))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (11) - 2, .025) and -0.056785776378734186 = (66.42573114549786 - 70.19776786012002) / 66.42573114549786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Commit range: dotnet/runtime@b9c1129...5eb82ac could be native pgo?

[EgorBo]

#20492 BDN related issue