[Perf] Windows/arm64: 14 Regressions on 4/26/2023 12:55:36 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetNow - Duration of single invocation	203.11 ns	243.23 ns	1.20	0.64	False	1162.9285705370626	1347.2544146813543	1.1585014323443499)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.GetNow

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -4.990244183938773 (T) = (0 -257.5767749388562) / Math.Sqrt((331.1234491697161 / (30)) + (180.332382988944 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -0.19533863113971525 = (215.48435583752982 - 257.5767749388562) / 215.48435583752982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	2.24 ns	5.93 ns	2.64	0.86	False	27.578813494346814	45.73479450682347	1.6583307514733483)	Trace	Trace
Parse - Duration of single invocation	14.15 ns	22.46 ns	1.59	0.79	False	169.59821852531314	177.2260565901561	1.0449759327142016)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -3.8660944898634786 (T) = (0 -4.877346444465968) / Math.Sqrt((0.4560155825639959 / (30)) + (1.2873655255535172 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -1.120305040840754 = (2.300304131018798 - 4.877346444465968) / 2.300304131018798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/24/2023 7:54:29 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -4.4729206225281635 (T) = (0 -22.39694616961747) / Math.Sqrt((8.345599139930595 / (26)) + (3.7693035438157403 / (6))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (6) - 2, .025) and -0.24157682431675248 = (18.039114238414243 - 22.39694616961747) / 18.039114238414243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Property_Get_int - Duration of single invocation	34.10 ns	47.77 ns	1.40	0.58	False	203.4601897689769	239.8109243697479	1.178662639811976)	Trace	Trace
Property_Set_class - Duration of single invocation	54.26 ns	68.44 ns	1.26	0.65	False	387.1508379888268	397.4716508438819	1.0266583766385982)	Trace	Trace
Property_Get_class - Duration of single invocation	21.49 ns	30.86 ns	1.44	0.70	False	172.29556288962232	214.3882175226586	1.2443049253682876)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Property_Get_int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.77184458010229 > 36.285741861998055.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -7.37598791202179 (T) = (0 -45.036374550275326) / Math.Sqrt((4.273495169991083 / (31)) + (5.614442980260123 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.30235069186193797 = (34.58083512505219 - 45.036374550275326) / 34.58083512505219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Reflection.Invoke.Property_Set_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -6.9632295888521645 (T) = (0 -71.32686795773019) / Math.Sqrt((27.588831853029323 / (30)) + (6.685966737969841 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -0.20950951606221513 = (58.97172945769635 - 71.32686795773019) / 58.97172945769635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Reflection.Invoke.Property_Get_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.863671275395028 > 23.933792222701896.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -5.590456219558039 (T) = (0 -31.929016994832086) / Math.Sqrt((2.6528625980392353 / (30)) + (6.841242327780629 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -0.3688917412965025 = (23.324720305925382 - 31.929016994832086) / 23.324720305925382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	18.31 ns	22.56 ns	1.23	0.70	False	116.41900666871472	136.46017565815322	1.1721468818787317)	Trace	Trace
ToString - Duration of single invocation	2.67 ns	5.67 ns	2.12	0.86	False	27.771337426090074	45.59131547982595	1.6416679823635254)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -8.824425205823863 (T) = (0 -23.40827573118519) / Math.Sqrt((4.531982924646435 / (28)) + (0.8447827026235848 / (3))) is less than -2.0452296421282288 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (3) - 2, .025) and -0.3351814725368086 = (17.531905746647382 - 23.40827573118519) / 17.531905746647382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -5.47682089739078 (T) = (0 -5.482289138784613) / Math.Sqrt((0.23711520283171592 / (29)) + (0.9174262168864238 / (3))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (3) - 2, .025) and -1.2716164956219647 = (2.413386744351657 - 5.482289138784613) / 2.413386744351657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse3 - Duration of single invocation	48.18 ns	86.00 ns	1.78	0.65	False	429.6916467764084	603.8758265980896	1.4053701791236324)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryParse3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/30/2023 4:24:24 AM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -5.01054248993185 (T) = (0 -81.97224605046725) / Math.Sqrt((29.07868606505789 / (30)) + (76.17701835860868 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -0.45737949544290246 = (56.24632863765907 - 81.97224605046725) / 56.24632863765907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.79 ns	6.64 ns	3.71	0.89	False	27.630785494351862	45.79467694658523	1.6573787580503758)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToString(value: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -4.606513284856507 (T) = (0 -5.6150202375542735) / Math.Sqrt((0.41856793056536695 / (30)) + (1.262434634343732 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -1.1783561972582712 = (2.577640995821283 - 5.6150202375542735) / 2.577640995821283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCustomAttributesMethodBaseMiss - Duration of single invocation	162.62 ns	220.30 ns	1.35	0.66	False	1087.181124501517	1136.695759932127	1.045544053621528)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -15.501712308109358 (T) = (0 -219.51745431387835) / Math.Sqrt((267.7757475277929 / (28)) + (2.1963199690678135 / (3))) is less than -2.0452296421282288 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (3) - 2, .025) and -0.29297016273653514 = (169.77766435791202 - 219.51745431387835) / 169.77766435791202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetLength - Duration of single invocation	39.32 ns	71.64 ns	1.82	0.72	False	151.56363857630924	215.13008777006172	1.419404349162204)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.GetLength(options: Read)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -12.118008097137519 (T) = (0 -71.49697941276683) / Math.Sqrt((17.74271872556614 / (29)) + (14.579486837780737 / (3))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (3) - 2, .025) and -0.6568990207535192 = (43.1510783199399 - 71.49697941276683) / 43.1510783199399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation	860.67 ns	1.37 μs	1.59	0.65	False	8302.531087829237	8795.066765578635	1.0593235571826296)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/21/2023 8:47:40 PM, 3/28/2023 2:42:21 AM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -4.689576812364017 (T) = (0 -1283.7375377025219) / Math.Sqrt((10294.941616773425 / (30)) + (11908.307038002367 / (3))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .025) and -0.3156132824778993 = (975.7711896041815 - 1283.7375377025219) / 975.7711896041815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.25094
Queue	AmpereWindows
Baseline	549077434782bd435574cebcf9ab37d6c9d611c2
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NewCustomConverter - Duration of single invocation	40.75 μs	53.54 μs	1.31	0.50	False	170796.37841832964	186025.69603356218	1.0891665136946378)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsRegressionStdDev: Marked as regression because -7.340913308585003 (T) = (0 -51931.993609179626) / Math.Sqrt((12566430.3944879 / (29)) + (1984512.6942444239 / (3))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (3) - 2, .025) and -0.17358063094836038 = (44250.89528549379 - 51931.993609179626) / 44250.89528549379 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Either dotnet/runtime#85355 or dotnet/runtime#85265