Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Regressions in System.Text.Perf_Utf8Encoding #68405

Closed
performanceautofiler bot opened this issue Apr 21, 2022 · 11 comments
Closed

[Perf] Regressions in System.Text.Perf_Utf8Encoding #68405

performanceautofiler bot opened this issue Apr 21, 2022 · 11 comments
Assignees
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI PGO runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Apr 21, 2022

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 38.88 μs 42.98 μs 1.11 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.97612073031136 > 40.968449278946196.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 3/22/2022 12:26:23 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -31.750718634657993 (T) = (0 -42956.59086093196) / Math.Sqrt((430473.58727225725 / (34)) + (1469.7221348553392 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.09153028452612605 = (39354.46544168128 - 42956.59086093196) / 39354.46544168128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 43.20 μs 57.55 μs 1.33 0.10 False
GetByteCount - Duration of single invocation 94.74 μs 126.51 μs 1.34 0.19 False
GetByteCount - Duration of single invocation 49.19 μs 62.90 μs 1.28 0.12 False
GetBytes - Duration of single invocation 245.95 μs 274.74 μs 1.12 0.15 False
GetByteCount - Duration of single invocation 37.50 μs 49.93 μs 1.33 0.12 False
GetBytes - Duration of single invocation 181.27 μs 219.02 μs 1.21 0.01 False
GetBytes - Duration of single invocation 261.76 μs 316.04 μs 1.21 0.00 False
GetBytes - Duration of single invocation 235.08 μs 301.14 μs 1.28 0.00 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.55368987438726 > 45.359647407631215.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -91.73236010708656 (T) = (0 -57550.17854516807) / Math.Sqrt((760051.9807085766 / (34)) + (74.63348954126008 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.31298297947779785 = (43831.6257291142 - 57550.17854516807) / 43831.6257291142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.50942006048385 > 99.47541709848485.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -83.80075313871093 (T) = (0 -126951.90332976988) / Math.Sqrt((2853991.9724543467 / (34)) + (374636.2011947571 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.32403998559926195 = (95882.22765969663 - 126951.90332976988) / 95882.22765969663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.89850403399311 > 49.58612193211858.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -91.63171970002061 (T) = (0 -62909.027349448814) / Math.Sqrt((887786.6067112109 / (33)) + (64.99075446288354 / (7))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (7) - 2, .025) and -0.31397229406357396 = (47876.98160278339 - 62909.027349448814) / 47876.98160278339 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 274.7420202432266 > 253.95162019782788.
IsChangePoint: Marked as a change because one of 1/20/2022 9:28:17 PM, 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -22.598811332621885 (T) = (0 -275170.4657499596) / Math.Sqrt((26557947.92640713 / (34)) + (10888849.200348983 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.1435628743132779 = (240625.56762801736 - 275170.4657499596) / 240625.56762801736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.93451808534916 > 39.376826979743136.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -8481.364447931946 (T) = (0 -49934.1444471577) / Math.Sqrt((8.318095246945353 / (34)) + (13.326755622261294 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.3314904552625797 = (37502.442657247826 - 49934.1444471577) / 37502.442657247826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.01502002314814 > 192.96708831532146.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -122.55527834589785 (T) = (0 -218875.21674137466) / Math.Sqrt((2201979.7565429383 / (34)) + (162724.6295646819 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.1992027101980438 = (182517.27992278157 - 218875.21674137466) / 182517.27992278157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 316.0359689999999 > 278.0157029962924.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -60.13173971568646 (T) = (0 -313438.849855305) / Math.Sqrt((5785687.545973331 / (34)) + (3523085.341259236 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.18685574527222645 = (264091.78293475945 - 313438.849855305) / 264091.78293475945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 301.1389536057692 > 246.46762090718286.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -99.96235018366377 (T) = (0 -301315.6104602944) / Math.Sqrt((11239783.433485491 / (34)) + (632285.9207757254 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.2742633916159605 = (236462.5810039008 - 301315.6104602944) / 236462.5810039008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.98 μs 2.37 μs 1.20 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.370236299618321 > 2.078712465202614.
IsChangePoint: Marked as a change because one of 2/11/2022 12:34:36 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -28.743781996752993 (T) = (0 -2370.4424267491386) / Math.Sqrt((5139.192095809534 / (33)) + (6.141634210601772 / (7))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (7) - 2, .025) and -0.17889588314060603 = (2010.7309395586528 - 2370.4424267491386) / 2010.7309395586528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

category:performance
theme:benchmarks

@performanceautofiler performanceautofiler bot added arm64 PGO untriaged New issue has not been triaged by the area owner labels Apr 21, 2022
@AndyAyersMS AndyAyersMS transferred this issue from dotnet/perf-autofiling-issues Apr 22, 2022
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Apr 22, 2022
@ghost
Copy link

ghost commented Apr 22, 2022

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 38.88 μs 42.98 μs 1.11 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.97612073031136 > 40.968449278946196.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 3/22/2022 12:26:23 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -31.750718634657993 (T) = (0 -42956.59086093196) / Math.Sqrt((430473.58727225725 / (34)) + (1469.7221348553392 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.09153028452612605 = (39354.46544168128 - 42956.59086093196) / 39354.46544168128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 43.20 μs 57.55 μs 1.33 0.10 False
GetByteCount - Duration of single invocation 94.74 μs 126.51 μs 1.34 0.19 False
GetByteCount - Duration of single invocation 49.19 μs 62.90 μs 1.28 0.12 False
GetBytes - Duration of single invocation 245.95 μs 274.74 μs 1.12 0.15 False
GetByteCount - Duration of single invocation 37.50 μs 49.93 μs 1.33 0.12 False
GetBytes - Duration of single invocation 181.27 μs 219.02 μs 1.21 0.01 False
GetBytes - Duration of single invocation 261.76 μs 316.04 μs 1.21 0.00 False
GetBytes - Duration of single invocation 235.08 μs 301.14 μs 1.28 0.00 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.55368987438726 > 45.359647407631215.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -91.73236010708656 (T) = (0 -57550.17854516807) / Math.Sqrt((760051.9807085766 / (34)) + (74.63348954126008 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.31298297947779785 = (43831.6257291142 - 57550.17854516807) / 43831.6257291142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.50942006048385 > 99.47541709848485.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -83.80075313871093 (T) = (0 -126951.90332976988) / Math.Sqrt((2853991.9724543467 / (34)) + (374636.2011947571 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.32403998559926195 = (95882.22765969663 - 126951.90332976988) / 95882.22765969663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.89850403399311 > 49.58612193211858.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -91.63171970002061 (T) = (0 -62909.027349448814) / Math.Sqrt((887786.6067112109 / (33)) + (64.99075446288354 / (7))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (7) - 2, .025) and -0.31397229406357396 = (47876.98160278339 - 62909.027349448814) / 47876.98160278339 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 274.7420202432266 > 253.95162019782788.
IsChangePoint: Marked as a change because one of 1/20/2022 9:28:17 PM, 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -22.598811332621885 (T) = (0 -275170.4657499596) / Math.Sqrt((26557947.92640713 / (34)) + (10888849.200348983 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.1435628743132779 = (240625.56762801736 - 275170.4657499596) / 240625.56762801736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.93451808534916 > 39.376826979743136.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 2/23/2022 2:01:04 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -8481.364447931946 (T) = (0 -49934.1444471577) / Math.Sqrt((8.318095246945353 / (34)) + (13.326755622261294 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.3314904552625797 = (37502.442657247826 - 49934.1444471577) / 37502.442657247826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.01502002314814 > 192.96708831532146.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -122.55527834589785 (T) = (0 -218875.21674137466) / Math.Sqrt((2201979.7565429383 / (34)) + (162724.6295646819 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.1992027101980438 = (182517.27992278157 - 218875.21674137466) / 182517.27992278157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 316.0359689999999 > 278.0157029962924.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -60.13173971568646 (T) = (0 -313438.849855305) / Math.Sqrt((5785687.545973331 / (34)) + (3523085.341259236 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.18685574527222645 = (264091.78293475945 - 313438.849855305) / 264091.78293475945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 301.1389536057692 > 246.46762090718286.
IsChangePoint: Marked as a change because one of 2/22/2022 8:48:40 PM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -99.96235018366377 (T) = (0 -301315.6104602944) / Math.Sqrt((11239783.433485491 / (34)) + (632285.9207757254 / (7))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (7) - 2, .025) and -0.2742633916159605 = (236462.5810039008 - 301315.6104602944) / 236462.5810039008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2c3e1c7fee778b1e58c0678eb9bac273fd700216
Compare d81ac24b2ba08751fe21689b5182806fea6b8735
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.98 μs 2.37 μs 1.20 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.370236299618321 > 2.078712465202614.
IsChangePoint: Marked as a change because one of 2/11/2022 12:34:36 AM, 4/16/2022 8:46:33 PM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsRegressionStdDev: Marked as regression because -28.743781996752993 (T) = (0 -2370.4424267491386) / Math.Sqrt((5139.192095809534 / (33)) + (6.141634210601772 / (7))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (7) - 2, .025) and -0.17889588314060603 = (2010.7309395586528 - 2370.4424267491386) / 2010.7309395586528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: -
Labels:

area-CodeGen-coreclr, untriaged, refs/heads/main, ubuntu 18.04, RunKind=micro, PGO, Regression, CoreClr, arm64

Milestone: -

@AndyAyersMS
Copy link
Member

Utf8 Encoding: None of our PGO training scenarios reach these code paths and so they aren't well optimized.

@DrewScoggins DrewScoggins changed the title [Perf] Changes at 4/17/2022 3:50:42 AM [Perf] Regressions in System.Text.Perf_Utf8Encoding Apr 22, 2022
@DrewScoggins DrewScoggins added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Apr 22, 2022
@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Apr 27, 2022
@JulieLeeMSFT JulieLeeMSFT added this to the 7.0.0 milestone Apr 27, 2022
@dakersnar
Copy link
Contributor

dakersnar commented Apr 28, 2022

Related windows arm64 regressions dotnet/perf-autofiling-issues#4730

@AndyAyersMS
Copy link
Member

@EgorBo if you are going to add non EN US PGO coverage, I would hope we'd see improvements here.

@lewing
Copy link
Member

lewing commented Jul 11, 2022

looks like #68055

@AndyAyersMS AndyAyersMS modified the milestones: 7.0.0, 8.0.0 Aug 11, 2022
@jozkee
Copy link
Member

jozkee commented Oct 14, 2022

Looking at the charts linked above, it looks like this regression was fixed around August 5th. Does anyone know what fixed the problem, and did that fix make it into RC2? The rc2 vs 6.0 perf report indicates that the regression is still present to some extent.

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Same 72613.69 72236.13 1.01 +0 ubuntu 18.04 Arm64 Unknown processor
Same 15235.54 14130.97 1.08 +0 Windows 11 Arm64 Unknown processor
Same 23516.05 21649.30 1.09 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Same 23697.14 21577.32 1.10 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 9988.59 20235.21 0.49 +0 macOS Monterey 12.6 Arm64 Apple M1
Slower 9768.82 20053.77 0.49 +0 macOS Monterey 12.6 Arm64 Apple M1 Max
Slower 9285.95 18929.47 0.49 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 5853.96 5720.55 1.02 +0 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 6004.45 6268.30 0.96 +0 Windows 11 X64 AMD Ryzen 9 5900X
Same 5122.50 5272.15 0.97 +0 Windows 11 X64 AMD Ryzen 9 7950X
Slower 7166.13 9885.03 0.72 +0 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower 7893.35 9209.80 0.86 +0 debian 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 5904.62 5823.80 1.01 +0 ubuntu 18.04 X64 AMD Ryzen 9 5900X
Same 9011.37 8110.84 1.11 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 5896.34 6096.48 0.97 +0 ubuntu 20.04 X64 AMD Ryzen 9 5900X
Slower 10756.76 59748.89 0.18 +0 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 7545.03 9296.18 0.81 +0 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) several?
Same 11532.30 11512.10 1.00 +0 macOS Big Sur 11.7 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
Same 11740.01 11658.43 1.01 +0 macOS Monterey 12.6 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Same 84786.32 87150.08 0.97 +0 ubuntu 18.04 Arm64 Unknown processor
Same 18038.05 16686.79 1.08 +0 Windows 11 Arm64 Unknown processor
Faster 33359.14 26622.22 1.25 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Faster 33353.04 26618.78 1.25 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 11567.60 23756.15 0.49 +0 macOS Monterey 12.6 Arm64 Apple M1
Slower 11255.32 23145.40 0.49 +0 macOS Monterey 12.6 Arm64 Apple M1 Max
Slower 11591.49 28018.07 0.41 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 6839.83 6729.10 1.02 +0 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 7036.56 7374.47 0.95 +0 Windows 11 X64 AMD Ryzen 9 5900X
Same 5891.89 5994.45 0.98 +0 Windows 11 X64 AMD Ryzen 9 7950X
Slower 8867.58 11569.86 0.77 +0 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) bimodal
Slower 8476.97 10668.41 0.79 +0 debian 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 6787.55 6700.24 1.01 +0 ubuntu 18.04 X64 AMD Ryzen 9 5900X
Same 10335.45 10344.95 1.00 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 6782.12 6962.38 0.97 +0 ubuntu 20.04 X64 AMD Ryzen 9 5900X
Slower 12203.78 38287.14 0.32 +0 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 8523.20 10614.18 0.80 +0 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 13175.46 13589.95 0.97 +0 macOS Big Sur 11.7 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
Same 11639.70 13373.55 0.87 +0 macOS Monterey 12.6 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell) bimodal

@AndyAyersMS
Copy link
Member

Suspect it might have been this change: #73973 where we updated the PGO training scenarios to cover other locales and added arm64 coverage.

Hard to tell exactly when because we had big outage right around then.

@AndyAyersMS
Copy link
Member

These do indeed seem to be fixed (though a few regressed again around 9/25), so closing. Random sampling:

newplot - 2022-11-01T075726 488

newplot - 2022-11-01T075559 012

@ghost ghost locked as resolved and limited conversation to collaborators Dec 1, 2022
@dakersnar
Copy link
Contributor

As of the 7 vs 6 report, it seems like this regression might still be present on Mac Arm64 configs. Could the fix have missed the 7.0 cutoff?

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Faster 1.37 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz bimodal
Slower 0.49 +0 macOS 13.0 Arm64 Apple M1
Same 0.98 +0 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 0.98 +0 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.75 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 1.02 +0 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 1.03 +0 Windows 11 X64 11th Gen Intel Core i9-11900 2.50GHz
Same 1.00 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 1.00 +0 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Same 1.00 +0 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Faster 1.16 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 0.50 +0 macOS 13.0 Arm64 Apple M1
Same 0.91 +0 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 0.91 +0 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.80 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 1.03 +0 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 0.90 +0 Windows 11 X64 11th Gen Intel Core i9-11900 2.50GHz
Same 1.10 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 1.01 +0 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Same 1.00 +0 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

@dakersnar dakersnar reopened this Dec 5, 2022
@EgorBo
Copy link
Member

EgorBo commented Dec 5, 2022

It's possible that PGO is simply not applied for any OSX (and the fix is in the PGO)

@jeffhandley jeffhandley added arch-arm64 runtime-coreclr specific to the CoreCLR runtime os-linux Linux OS (any supported distro) and removed arm64 labels Dec 28, 2022
@BruceForstall BruceForstall removed arch-arm64 os-linux Linux OS (any supported distro) labels Jan 23, 2023
@AndyAyersMS
Copy link
Member

Closing this since we've now enabled dynamic PGO (and spot checking, perf for these cases looks good).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI PGO runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Development

No branches or pull requests

9 participants