.NET 7.0 Preview 7 Microbenchmarks Performance Study Report #73866
Labels
area-Meta
tenet-performance
Performance related issue
tenet-performance-benchmarks
Issue from performance benchmark
tracking
This issue is tracking the completion of other related issues.
Milestone
Comments
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
ghost
added
the
mrsharm
added
tenet-performance
|
Tagging subscribers to this area: @dotnet/area-meta Issue DetailsDataThis time we have covered following configurations comparing .NET 7.0 Preview 7 vs. .NET 7.0 Preview 5:
Most of the benchmarks were run on bare-metal machines, some were executed via WSL. This would not be possible without the help from: @adamsitnik , @AndyAyersMS, @brianrob, @jozkee , @ivanpovazan, @kunalspathak, @jeffhandley, @GrabYourPitchforks, @tannergooding who contributed their results and time. An addtional thank you to @dakersnar for shadowing the creation of this report and helping with the vetting of the results! The full report generated by the tool is available here. You will have to click "Raw" to see the entire file. The report is sorted from most regressed to most improved, so scroll to the bottom in the full report to see improvements. There are plenty of them! Again, the full historical data turned out to be extremely useful. For details about methodology please read #41871. Preview 5 report can be found here. Notable Improvements
RegressionsBy Design
Investigation In Progress
Already Solved
Noise, Flaky or MultimodalThe following benchmarks showed up in the report generated by the tool, but were not actual regressions:
StatisticsTotal: 119052 Statistics per Architecture
Statistics per Operating System
Statistics per Namespace
Big thanks to everyone involved!
|
|
setting milestone 7, but if all the "investigating" issues have something tracking them (for 7 or later) or are determined not worth tracking, presumably we can close this. |
|
Usually we close these once the next report gets released (which should be in the next couple days). Feel free to close it now if it makes things easier. |
|
@dakersnar iff it tracks work necessary to ship .NET 7, it should stay open and in 7.0.0 milestone. If not, something else (eg Future or closed) My assumption is that any work would already be tracked by individual issues linked above but that wasn't entirely clear. |
|
Your assumption is correct. I think I'll move this to "Future" for now. Any individual issues within this report that are targeting 7.0 should already be individually marked as such. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Data
This time we have covered following configurations comparing .NET 7.0 Preview 7 vs. .NET 7.0 Preview 5:
Most of the benchmarks were run on bare-metal machines, some were executed via WSL.
This would not be possible without the help from: @adamsitnik , @AndyAyersMS, @brianrob, @jozkee , @ivanpovazan, @kunalspathak, @jeffhandley, @GrabYourPitchforks, @tannergooding who contributed their results and time. An addtional thank you to @dakersnar for shadowing the creation of this report and helping with the vetting of the results!
The full report generated by the tool is available here. You will have to click "Raw" to see the entire file. The report is sorted from most regressed to most improved, so scroll to the bottom in the full report to see improvements. There are plenty of them!
Again, the full historical data turned out to be extremely useful. For details about methodology please read #41871. Preview 5 report can be found here.
Notable Improvements
System.Perf_Convert.ToBase64String(formattingOptions: None)System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmarkTYP_STRUCTLCL_VAR/LCL_FLDcall args on Windows x64 #70777Benchstone.MDBenchF.MDInvMt.TestSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)System.Numerics.Tests.Perf_VectorConvert.Convert_ulong_doubleSystem.Tests.Perf_Environment.GetLogicalDrivesRegressions
By Design
Benchstone.BenchI.Array2.TestSystem.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000)System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None),System.Collections.IterateForEach<Int32>.HashSet(Size: 512),System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)),System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)),System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (enUS, IgnoreCase, False)),System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (enUS, IgnoreCase, False)),System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)),System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False)),System.IO.Tests.Perf_Path.GetDirectoryName,System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable),System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308"),System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308"),System.Tests.Perf_Double.Parse(value: "12345"),System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308"),System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308"),System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615"),System.Tests.Perf_UInt16.TryParse(value: "12345"),System.Tests.Perf_Double.TryParse(value: "12345"),System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308"),System.IO.Tests.Perf_Path.GetDirectoryNameSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking)System.Security.Cryptography.X509Certificates.Tests.X509ChainTests.BuildX509ChainContoso,System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled),System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled)System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")Investigation In Progress
Benchstone.MDBenchI.MDMulMatrix.TestSystem.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPongSystem.Collections.IndexerSet<String>.Array(Size: 512)System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True),System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False),System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False),System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False),System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmarkSystem.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromYieldingAsyncMethodSystem.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))System.IO.Tests.Perf_File.DeleteAlready Solved
System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverterSystem.Tests.Perf_UInt64.Parse(value: "12345"),System.Tests.Perf_UInt16.TryParse(value: "12345")Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgumentNoise, Flaky or Multimodal
The following benchmarks showed up in the report generated by the tool, but were not actual regressions:
CscBench.CompileTestMicrosoft.Extensions.Primitives.Performance.StringValuesBenchmark.Indexer_FirstElement_StringSystem.Numerics.Tests.Perf_VectorOf<Int32>.ConditionalSelectBenchmarkSystem.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmarkSystem.Numerics.Tests.Perf_Vector4.LengthSquaredBenchmarkSystem.Numerics.Tests.Perf_Vector4.SquareRootBenchmarkSystem.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmarkSystem.Numerics.Tests.Perf_VectorOf<*>.*System.Memory.Span<Int32>.StartsWith(Size: 512)PerfLabTests.CastingPerf.CheckArrayIsArrayByVarianceSystem.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)PerfLabTests.StackWalk.WalkSystem.Memory.MemoryMarshal<Byte>.CastToByteSystem.Memory.MemoryMarshal<Byte>.CastToIntSystem.Memory.MemoryMarshal<Byte>.ReadSystem.Memory.MemoryMarshal<Int32>.ReadSystem.Numerics.Tests.Constructor.ConstructorBenchmark_SingleSystem.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegmentsSystem.Diagnostics.Perf_Process.GetProcessesByNameSystem.Diagnostics.Perf_Process.GetProcessesStatistics
Total: 119052
Same: 77.32 %
Slower: 4.81 %
Faster: 8.99 %
Noise: 8.83 %
Unknown: 0.04 %
Statistics per Architecture
Statistics per Operating System
Statistics per Namespace
Big thanks to everyone involved!
The text was updated successfully, but these errors were encountered: