Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use file preallocation on ZipFileExtensions.ExtractToFile #51549

Closed
wants to merge 8 commits into from

Conversation

jozkee
Copy link
Member

@jozkee jozkee commented Apr 20, 2021

Contributes to #51116; builds on top of #51111. The only relevant change in this PR is 12d7f65.

Results obtained running the ExtractEntryToFile benchmarks added in dotnet/performance#1781:
Note: Performance slightly improves in all the cases with gains of ~6%-1%.

BenchmarkDotNet=v0.12.1.1521-nightly, OS=Windows 10.0.19042.928 (20H2/October2020Update)
Intel Core i7-9750H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=6.0.100-preview.3.21207.34
  [Host]     : .NET 6.0.0 (6.0.21.20104), X64 RyuJIT
  Job-BMCEYP : .NET 6.0.0 (42.42.42.42424), X64 RyuJIT
  Job-VSGHKS : .NET 6.0.0 (42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  

|             Method | Toolchain |   TestFolder | CompressionLevel |       Mean |     Error |    StdDev |     Median |        Min |        Max | Ratio | Allocated |
|------------------- |---------- |------------- |----------------- |-----------:|----------:|----------:|-----------:|-----------:|-----------:|------:|----------:|
| ExtractEntryToFile | base      | TestDocument |          Optimal | 1,535.6 μs |  90.76 μs | 104.52 μs | 1,523.8 μs | 1,369.4 μs | 1,717.8 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |          Optimal | 1,508.3 μs |  68.67 μs |  79.08 μs | 1,520.1 μs | 1,376.7 μs | 1,685.7 μs |  0.99 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |          Fastest | 1,504.4 μs |  78.66 μs |  90.59 μs | 1,500.6 μs | 1,369.9 μs | 1,660.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |          Fastest | 1,486.4 μs |  73.73 μs |  84.91 μs | 1,493.6 μs | 1,344.0 μs | 1,636.4 μs |  0.99 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |    NoCompression | 2,918.2 μs | 140.14 μs | 161.38 μs | 2,946.2 μs | 2,600.2 μs | 3,136.0 μs |  1.00 |     374 B |
| ExtractEntryToFile |  new      | TestDocument |    NoCompression | 2,769.4 μs | 105.90 μs | 121.95 μs | 2,783.6 μs | 2,523.8 μs | 2,912.3 μs |  0.95 |     502 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |     SmallestSize | 1,525.4 μs | 113.56 μs | 126.22 μs | 1,479.0 μs | 1,318.0 μs | 1,761.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |     SmallestSize | 1,465.9 μs |  84.08 μs |  96.83 μs | 1,489.6 μs | 1,324.5 μs | 1,604.5 μs |  0.97 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |          Optimal | 1,598.7 μs |  55.80 μs |  64.26 μs | 1,581.4 μs | 1,491.0 μs | 1,719.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |          Optimal | 1,538.2 μs |  78.91 μs |  90.87 μs | 1,520.2 μs | 1,376.7 μs | 1,687.7 μs |  0.96 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |          Fastest | 1,701.1 μs |  70.74 μs |  81.46 μs | 1,713.7 μs | 1,529.3 μs | 1,847.7 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |          Fastest | 1,636.8 μs |  82.96 μs |  95.54 μs | 1,608.0 μs | 1,479.9 μs | 1,836.5 μs |  0.96 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |    NoCompression | 2,454.6 μs | 135.28 μs | 150.36 μs | 2,478.4 μs | 1,957.3 μs | 2,678.5 μs |  1.00 |     374 B |
| ExtractEntryToFile |  new      |      alice29 |    NoCompression | 2,392.5 μs | 112.77 μs | 129.87 μs | 2,389.6 μs | 2,189.4 μs | 2,646.5 μs |  0.98 |     502 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |     SmallestSize | 1,613.6 μs |  63.51 μs |  70.59 μs | 1,602.2 μs | 1,451.1 μs | 1,738.6 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |     SmallestSize | 1,535.9 μs |  72.42 μs |  80.49 μs | 1,538.3 μs | 1,392.9 μs | 1,655.5 μs |  0.95 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |          Optimal |   996.7 μs |  63.69 μs |  73.34 μs | 1,014.3 μs |   860.5 μs | 1,100.4 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |          Optimal |   977.9 μs |  55.90 μs |  62.13 μs |   954.6 μs |   889.2 μs | 1,110.6 μs |  0.99 |     778 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |          Fastest | 1,007.4 μs |  49.65 μs |  57.18 μs |   989.3 μs |   897.2 μs | 1,100.4 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |          Fastest |   993.5 μs |  57.22 μs |  63.60 μs |   984.5 μs |   858.3 μs | 1,100.2 μs |  0.99 |     779 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |    NoCompression | 1,990.3 μs |  70.37 μs |  81.04 μs | 1,991.1 μs | 1,848.3 μs | 2,118.7 μs |  1.00 |     373 B |
| ExtractEntryToFile |  new      |          sum |    NoCompression | 1,872.5 μs | 118.22 μs | 136.14 μs | 1,906.7 μs | 1,590.4 μs | 2,073.9 μs |  0.94 |     501 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |     SmallestSize | 1,008.7 μs |  87.69 μs | 100.98 μs |   987.2 μs |   853.5 μs | 1,216.7 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |     SmallestSize |   978.3 μs |  34.97 μs |  37.42 μs |   986.1 μs |   903.1 μs | 1,056.0 μs |  0.98 |     778 B |

@dotnet-issue-labeler
Copy link

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost
Copy link

ghost commented Apr 20, 2021

Tagging subscribers to this area: @carlossanlop
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #51116; builds on top of #51111. The only relevant change in this PR is 12d7f65.

Results obtained running the ExtractEntryToFile benchmarks added in dotnet/performance#1781:
Note: Performance slightly improves in all the cases with gains of ~6%-1%.

BenchmarkDotNet=v0.12.1.1521-nightly, OS=Windows 10.0.19042.928 (20H2/October2020Update)
Intel Core i7-9750H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=6.0.100-preview.3.21207.34
  [Host]     : .NET 6.0.0 (6.0.21.20104), X64 RyuJIT
  Job-BMCEYP : .NET 6.0.0 (42.42.42.42424), X64 RyuJIT
  Job-VSGHKS : .NET 6.0.0 (42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  

|             Method | Toolchain |   TestFolder | CompressionLevel |       Mean |     Error |    StdDev |     Median |        Min |        Max | Ratio | Allocated |
|------------------- |---------- |------------- |----------------- |-----------:|----------:|----------:|-----------:|-----------:|-----------:|------:|----------:|
| ExtractEntryToFile | base      | TestDocument |          Optimal | 1,535.6 μs |  90.76 μs | 104.52 μs | 1,523.8 μs | 1,369.4 μs | 1,717.8 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |          Optimal | 1,508.3 μs |  68.67 μs |  79.08 μs | 1,520.1 μs | 1,376.7 μs | 1,685.7 μs |  0.99 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |          Fastest | 1,504.4 μs |  78.66 μs |  90.59 μs | 1,500.6 μs | 1,369.9 μs | 1,660.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |          Fastest | 1,486.4 μs |  73.73 μs |  84.91 μs | 1,493.6 μs | 1,344.0 μs | 1,636.4 μs |  0.99 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |    NoCompression | 2,918.2 μs | 140.14 μs | 161.38 μs | 2,946.2 μs | 2,600.2 μs | 3,136.0 μs |  1.00 |     374 B |
| ExtractEntryToFile |  new      | TestDocument |    NoCompression | 2,769.4 μs | 105.90 μs | 121.95 μs | 2,783.6 μs | 2,523.8 μs | 2,912.3 μs |  0.95 |     502 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      | TestDocument |     SmallestSize | 1,525.4 μs | 113.56 μs | 126.22 μs | 1,479.0 μs | 1,318.0 μs | 1,761.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      | TestDocument |     SmallestSize | 1,465.9 μs |  84.08 μs |  96.83 μs | 1,489.6 μs | 1,324.5 μs | 1,604.5 μs |  0.97 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |          Optimal | 1,598.7 μs |  55.80 μs |  64.26 μs | 1,581.4 μs | 1,491.0 μs | 1,719.0 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |          Optimal | 1,538.2 μs |  78.91 μs |  90.87 μs | 1,520.2 μs | 1,376.7 μs | 1,687.7 μs |  0.96 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |          Fastest | 1,701.1 μs |  70.74 μs |  81.46 μs | 1,713.7 μs | 1,529.3 μs | 1,847.7 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |          Fastest | 1,636.8 μs |  82.96 μs |  95.54 μs | 1,608.0 μs | 1,479.9 μs | 1,836.5 μs |  0.96 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |    NoCompression | 2,454.6 μs | 135.28 μs | 150.36 μs | 2,478.4 μs | 1,957.3 μs | 2,678.5 μs |  1.00 |     374 B |
| ExtractEntryToFile |  new      |      alice29 |    NoCompression | 2,392.5 μs | 112.77 μs | 129.87 μs | 2,389.6 μs | 2,189.4 μs | 2,646.5 μs |  0.98 |     502 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |      alice29 |     SmallestSize | 1,613.6 μs |  63.51 μs |  70.59 μs | 1,602.2 μs | 1,451.1 μs | 1,738.6 μs |  1.00 |     652 B |
| ExtractEntryToFile |  new      |      alice29 |     SmallestSize | 1,535.9 μs |  72.42 μs |  80.49 μs | 1,538.3 μs | 1,392.9 μs | 1,655.5 μs |  0.95 |     780 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |          Optimal |   996.7 μs |  63.69 μs |  73.34 μs | 1,014.3 μs |   860.5 μs | 1,100.4 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |          Optimal |   977.9 μs |  55.90 μs |  62.13 μs |   954.6 μs |   889.2 μs | 1,110.6 μs |  0.99 |     778 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |          Fastest | 1,007.4 μs |  49.65 μs |  57.18 μs |   989.3 μs |   897.2 μs | 1,100.4 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |          Fastest |   993.5 μs |  57.22 μs |  63.60 μs |   984.5 μs |   858.3 μs | 1,100.2 μs |  0.99 |     779 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |    NoCompression | 1,990.3 μs |  70.37 μs |  81.04 μs | 1,991.1 μs | 1,848.3 μs | 2,118.7 μs |  1.00 |     373 B |
| ExtractEntryToFile |  new      |          sum |    NoCompression | 1,872.5 μs | 118.22 μs | 136.14 μs | 1,906.7 μs | 1,590.4 μs | 2,073.9 μs |  0.94 |     501 B |
|                    |           |              |                  |            |           |           |            |            |            |       |           |
| ExtractEntryToFile | base      |          sum |     SmallestSize | 1,008.7 μs |  87.69 μs | 100.98 μs |   987.2 μs |   853.5 μs | 1,216.7 μs |  1.00 |     650 B |
| ExtractEntryToFile |  new      |          sum |     SmallestSize |   978.3 μs |  34.97 μs |  37.42 μs |   986.1 μs |   903.1 μs | 1,056.0 μs |  0.98 |     778 B |
Author: Jozkee
Assignees: -
Labels:

area-System.IO, new-api-needs-documentation

Milestone: -

@danmoseley
Copy link
Member

Is it surprising that 'fastest' is slower than 'optimal' in 2 of the 3 cases? Is that zlib wierdness, or on our side somehow?

@ghost ghost closed this May 21, 2021
@ghost
Copy link

ghost commented May 21, 2021

Draft Pull Request was automatically closed for inactivity. Please let us know if you'd like to reopen it.

@ghost ghost locked as resolved and limited conversation to collaborators Jun 20, 2021
@jozkee jozkee deleted the preallocate_zip branch April 16, 2024 15:47
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants