[Perf] Changes at 2/3/2022 10:14:29 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	bc76b4e22ccb690db1d0373a4fad11679e2ead46
Compare	8b9416581ed4db6e283bb8a0eb410994ad390f82
Diff	Diff

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadByte - Duration of single invocation	24.02 μs	28.41 μs	1.18	0.11	False
OpenClose - Duration of single invocation	15.87 μs	25.19 μs	1.59	0.16	False
SeekForward - Duration of single invocation	26.64 μs	30.25 μs	1.14	0.07	False
OpenClose - Duration of single invocation	14.75 μs	18.10 μs	1.23	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.4054718693285 > 25.503363131487085.
IsChangePoint: Marked as a change because one of 12/3/2021 5:08:27 AM, 12/8/2021 1:12:03 PM, 12/13/2021 6:35:57 PM, 1/24/2022 12:53:54 PM, 2/3/2022 7:13:55 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -17.926459451082852 (T) = (0 -28237.18950153167) / Math.Sqrt((871487.0389387782 / (54)) + (708249.4599843788 / (24))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (54) + (24) - 2, .025) and -0.15692582019542717 = (24407.0873072587 - 28237.18950153167) / 24407.0873072587 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.OpenClose(fileSize: 1024, options: Asynchronous)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.1870576715289 > 16.650344567850365.
IsChangePoint: Marked as a change because one of 12/3/2021 3:02:06 AM, 12/8/2021 1:12:03 PM, 12/13/2021 11:29:53 PM, 1/24/2022 12:53:54 PM, 2/3/2022 7:13:55 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -16.564759863341894 (T) = (0 -19985.305687592358) / Math.Sqrt((68288.45352196546 / (54)) + (1337537.6655580383 / (24))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (54) + (24) - 2, .025) and -0.24669066779040374 = (16030.685240480463 - 19985.305687592358) / 16030.685240480463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.24797066408768 > 28.063978775597274.
IsChangePoint: Marked as a change because one of 12/3/2021 3:02:06 AM, 12/8/2021 1:12:03 PM, 12/13/2021 11:29:53 PM, 1/24/2022 12:53:54 PM, 2/3/2022 10:14:29 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -33.90162231361112 (T) = (0 -30583.670211202527) / Math.Sqrt((73992.04115299533 / (55)) + (246553.22252801593 / (23))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (23) - 2, .025) and -0.13863734729144173 = (26859.886761973947 - 30583.670211202527) / 26859.886761973947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.OpenClose(fileSize: 1024, options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.09808093869732 > 15.21760646638086.
IsChangePoint: Marked as a change because one of 12/3/2021 5:08:27 AM, 12/8/2021 8:10:20 AM, 12/14/2021 12:54:03 AM, 1/24/2022 12:53:54 PM, 2/4/2022 1:40:45 AM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -14.959579072246656 (T) = (0 -18273.647471944903) / Math.Sqrt((119034.5648704398 / (56)) + (1337449.1494157207 / (22))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (56) + (22) - 2, .025) and -0.2584077947189638 = (14521.244662208961 - 18273.647471944903) / 14521.244662208961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	bc76b4e22ccb690db1d0373a4fad11679e2ead46
Compare	8b9416581ed4db6e283bb8a0eb410994ad390f82
Diff	Diff

Regressions in System.IO.Tests.Perf_Directory

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Exists - Duration of single invocation	10.70 μs	15.99 μs	1.49	0.22	False
RecursiveCreateDeleteDirectory - Duration of single invocation	24.77 ms	27.52 ms	1.11	0.08	False
CreateDirectory - Duration of single invocation	120.15 μs	141.93 μs	1.18	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Directory.Exists

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.987613173957277 > 11.424876586370743.
IsChangePoint: Marked as a change because one of 12/3/2021 3:02:06 AM, 12/8/2021 1:12:03 PM, 12/13/2021 6:35:57 PM, 1/24/2022 12:53:54 PM, 2/3/2022 7:13:55 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -21.72213400096961 (T) = (0 -16168.648411004586) / Math.Sqrt((49089.5377434706 / (54)) + (1343996.334523113 / (24))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (54) + (24) - 2, .025) and -0.4716557720144727 = (10986.705395700088 - 16168.648411004586) / 10986.705395700088 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Directory.RecursiveCreateDeleteDirectory(depth: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.515006666666675 > 25.4252943.
IsChangePoint: Marked as a change because one of 12/3/2021 3:02:06 AM, 12/8/2021 1:12:03 PM, 12/13/2021 6:35:57 PM, 1/24/2022 12:53:54 PM, 2/4/2022 1:40:45 AM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -20.224331725790964 (T) = (0 -26738476.7999223) / Math.Sqrt((208483949109.42688 / (56)) + (285585276118.6987 / (22))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (56) + (22) - 2, .025) and -0.10834910487535657 = (24124598.18148116 - 26738476.7999223) / 24124598.18148116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Directory.CreateDirectory

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 141.9292857142857 > 128.85933846153847.
IsChangePoint: Marked as a change because one of 12/3/2021 5:08:27 AM, 12/8/2021 1:12:03 PM, 12/13/2021 11:29:53 PM, 1/24/2022 12:53:54 PM, 2/4/2022 1:40:45 AM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -13.967770301051566 (T) = (0 -141207.25274725276) / Math.Sqrt((16813282.31452239 / (56)) + (35396881.0136802 / (22))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (56) + (22) - 2, .025) and -0.15831451567887303 = (121907.52238349791 - 141207.25274725276) / 121907.52238349791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

Dup of dotnet/runtime#64989