[Perf] Linux/arm64: 12 Regressions on 2/2/2023 8:18:56 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	3c47b2c75dfca5a82a59db238f103cdbda902826
Compare	873775b42431bf27a9ae69633864e5290443c29b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
MultiplyByQuaternionOperatorBenchmark - Duration of single invocation	2.22 ns	7.94 ns	3.57	0.36	False
LerpBenchmark - Duration of single invocation	14.14 ns	36.98 ns	2.62	0.01	False
SlerpBenchmark - Duration of single invocation	14.12 ns	37.19 ns	2.63	0.03	False
CreateFromRotationMatrixBenchmark - Duration of single invocation	15.51 ns	17.74 ns	1.14	0.05	False
DivisionOperatorBenchmark - Duration of single invocation	8.90 ns	15.65 ns	1.76	0.01	False
ConcatenateBenchmark - Duration of single invocation	2.20 ns	8.08 ns	3.68	0.39	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.941581479533306 > 2.3212239364452296.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -167.45954392890064 (T) = (0 -8.105748291406591) / Math.Sqrt((0.009554647809051419 / (24)) + (0.013660284774689476 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -2.7170171732207136 = (2.180713166945941 - 8.105748291406591) / 2.180713166945941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.983729453513575 > 14.878670792138404.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -655.3871681404464 (T) = (0 -37.14380633354826) / Math.Sqrt((0.014716382798237213 / (24)) + (0.009806419615714195 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -1.6166620942734653 = (14.195110027709367 - 37.14380633354826) / 14.195110027709367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.SlerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.194904381975675 > 14.831858502405032.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -681.314249430754 (T) = (0 -37.12840737317261) / Math.Sqrt((0.01145128982304304 / (24)) + (0.010573844010070088 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -1.6248887137268486 = (14.144754853419004 - 37.12840737317261) / 14.144754853419004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.CreateFromRotationMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.74071543547166 > 16.523233185803527.
IsChangePoint: Marked as a change because one of 1/6/2023 8:18:17 PM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -66.16114680045021 (T) = (0 -17.685975274609746) / Math.Sqrt((0.006977108161075092 / (24)) + (0.009905508520053573 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.1271876354155651 = (15.69035599657672 - 17.685975274609746) / 15.69035599657672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.DivisionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.652727299059364 > 9.351297146848694.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -179.15302155289265 (T) = (0 -15.597904040419898) / Math.Sqrt((0.010926303958308299 / (24)) + (0.015376183196756387 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -0.7613279646593847 = (8.85576357917891 - 15.597904040419898) / 8.85576357917891 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.082194213649098 > 2.3080874927400212.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -158.75556827975439 (T) = (0 -8.044385993022145) / Math.Sqrt((0.005161299902007397 / (23)) + (0.018279518290545582 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -2.6985814767956176 = (2.174992235128926 - 8.044385993022145) / 2.174992235128926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	3c47b2c75dfca5a82a59db238f103cdbda902826
Compare	873775b42431bf27a9ae69633864e5290443c29b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TransformBenchmark - Duration of single invocation	24.31 ns	28.59 ns	1.18	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.58503990700628 > 24.11130244544981.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -24.07581338992146 (T) = (0 -28.66686478868555) / Math.Sqrt((1.2286013034574401 / (23)) + (0.014949985175678191 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.24346838304287843 = (23.05395551637201 - 28.66686478868555) / 23.05395551637201 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	3c47b2c75dfca5a82a59db238f103cdbda902826
Compare	873775b42431bf27a9ae69633864e5290443c29b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation	0.79 ns	7.38 ns	9.36	0.33	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.3838777325075755 > 0.8277504860603464.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -126.99526607522674 (T) = (0 -7.35893009654001) / Math.Sqrt((0.05410990308401292 / (24)) + (0.00642522359262174 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -8.042009880592124 = (0.8138599928247484 - 7.35893009654001) / 0.8138599928247484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	3c47b2c75dfca5a82a59db238f103cdbda902826
Compare	873775b42431bf27a9ae69633864e5290443c29b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation	0.04 ns	2.46 ns	55.28	0.78	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector2.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4592848641998617 > 0.003390660399691646.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -27.399889821767825 (T) = (0 -2.5765840624988376) / Math.Sqrt((0.1773866162467119 / (24)) + (0.011222966552333794 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -22.060985317555495 = (0.11172914023484405 - 2.5765840624988376) / 0.11172914023484405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	3c47b2c75dfca5a82a59db238f103cdbda902826
Compare	873775b42431bf27a9ae69633864e5290443c29b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation	1.34 ns	8.04 ns	6.02	0.29	False
TransformVector2ByQuaternionBenchmark - Duration of single invocation	0.49 ns	5.80 ns	11.93	0.40	False
TransformVector3ByQuaternionBenchmark - Duration of single invocation	1.30 ns	8.04 ns	6.17	0.26	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.038862144980559 > 1.375150017228637.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -253.67002250969566 (T) = (0 -8.003021323808895) / Math.Sqrt((0.0031426920369748467 / (24)) + (0.00903622795642789 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -5.09920332347962 = (1.3121420781301547 - 8.003021323808895) / 1.3121420781301547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.804749575425005 > 0.504518725267644.
IsChangePoint: Marked as a change because one of 12/28/2022 7:19:51 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -168.98161044721124 (T) = (0 -5.964098047930127) / Math.Sqrt((0.011060216079014102 / (24)) + (0.009670498095240392 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -12.28745338259838 = (0.44885185115613474 - 5.964098047930127) / 0.44885185115613474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.037643934677403 > 1.369175346531895.
IsChangePoint: Marked as a change because one of 1/22/2023 2:59:30 AM, 2/2/2023 2:17:49 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM.
IsRegressionStdDev: Marked as regression because -196.70057792976218 (T) = (0 -7.9674953362402965) / Math.Sqrt((0.011146751982089 / (24)) + (0.01126782603079823 / (16))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (16) - 2, .025) and -5.409497985068502 = (1.2430763461976722 - 7.9674953362402965) / 1.2430763461976722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Looks like dotnet/runtime#81335