[Perf] Linux/x64: 4 Regressions on 9/27/2022 4:04:11 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor_Bytes - Duration of single invocation	389.18 ns	793.04 ns	2.04	0.10	False
GetAddressBytes - Duration of single invocation	443.87 ns	727.40 ns	1.64	0.08	False
Ctor_Span - Duration of single invocation	365.29 ns	755.99 ns	2.07	0.06	False
TryWriteBytes - Duration of single invocation	434.88 ns	735.09 ns	1.69	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 793.0396262103835 > 412.97496375910043.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsRegressionStdDev: Marked as regression because -148.05747465230505 (T) = (0 -783.4900374817038) / Math.Sqrt((79.9783756712201 / (29)) + (149.15722185030273 / (36))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (36) - 2, .025) and -0.9858166470640966 = (394.5429899784776 - 783.4900374817038) / 394.5429899784776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 727.4031845899871 > 466.349269077429.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsRegressionStdDev: Marked as regression because -70.33021988145737 (T) = (0 -748.3544321021982) / Math.Sqrt((25.334552521454054 / (29)) + (617.3962515784395 / (35))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (35) - 2, .025) and -0.6788944995741768 = (445.74238124671064 - 748.3544321021982) / 445.74238124671064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 755.9905045813417 > 384.91259737238437.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsRegressionStdDev: Marked as regression because -128.06071790809835 (T) = (0 -762.6907463732886) / Math.Sqrt((39.46128765588628 / (30)) + (259.8667370064361 / (32))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (32) - 2, .025) and -1.0651860876414894 = (369.3084855342535 - 762.6907463732886) / 369.3084855342535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 735.0905385824451 > 455.8016923746357.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsRegressionStdDev: Marked as regression because -65.85639938115801 (T) = (0 -736.6458241790391) / Math.Sqrt((206.4693472473326 / (29)) + (450.4780517475538 / (34))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (34) - 2, .025) and -0.6763948337033375 = (439.4226284697554 - 736.6458241790391) / 439.4226284697554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Min - Duration of single invocation	2.47 μs	2.16 μs	0.88	0.26	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1593931241785715 < 2.35453123163365.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsImprovementStdDev: Marked as improvement because 12.098051343033308 (T) = (0 -2187.096992844359) / Math.Sqrt((13836.091486949126 / (299)) + (6580.853245694546 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.07890469605020306 = (2374.452440986024 - 2187.096992844359) / 2374.452440986024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Improvements in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor_Span - Duration of single invocation	37.93 ns	24.26 ns	0.64	0.52	False
Ctor_Bytes - Duration of single invocation	39.22 ns	33.62 ns	0.86	0.52	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.2637587990092 < 32.6342076654047.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsImprovementStdDev: Marked as improvement because 12.658821531369549 (T) = (0 -27.948175501777232) / Math.Sqrt((7.392196679247289 / (299)) + (5.986580804515473 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17056751068837545 = (33.695539856381096 - 27.948175501777232) / 33.695539856381096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.61754181625243 < 37.067326680794366.
IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM.
IsImprovementStdDev: Marked as improvement because 11.083034367972369 (T) = (0 -34.11393548793784) / Math.Sqrt((11.614310841847756 / (299)) + (3.8175924939423656 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11324863453648766 = (38.4706884213324 - 34.11393548793784) / 38.4706884213324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[naricc]

@vargaz There are several improvements and a few substantial regressions with this change: dotnet/runtime@c7c6aa0. Is there anything to do about the regressions?

[lewing]

@naricc I think the ipaddress regressions are probably from dotnet/runtime#75872