[wasm] Fix perf pipeline errors #75406

radical · 2022-09-11T00:53:32Z

Build SizeOnDisk, and Startup tools for `net7.0` instead of `net6.0`.

It current fails with:

error NU1102: Unable to find package Microsoft.NETCore.App.Ref with version (= 6.0.10)
error NU1102:   - Found 2073 version(s) in dotnet6 [ Nearest version: 6.0.2-servicing.1.22101.5 ]
error NU1102:   - Found 1520 version(s) in dotnet7 [ Nearest version: 7.0.0-alpha.1.21417.28 ]
error NU1102:   - Found 1120 version(s) in dotnet5 [ Nearest version: 6.0.0-alpha.1.20420.3 ]
error NU1102:   - Found 76 version(s) in dotnet3.1 [ Nearest version: 3.1.1-servicing.19602.11 ]
error NU1102:   - Found 52 version(s) in dotnet-public [ Nearest version: 7.0.0-preview.1.22076.8 ]
error NU1102:   - Found 1 version(s) in dotnet-eng [ Nearest version: 5.0.0-alpha.1.19618.1 ]
error NU1102:   - Found 0 version(s) in benchmark-dotnet-prerelease
error NU1102:   - Found 0 version(s) in dotnet-tools
error NU1102:   - Found 0 version(s) in dotnet3.1-transport
error NU1102:   - Found 0 version(s) in dotnet5-transport

We are building with 8.0 sdk now, but for tfm=net7.0 and that confuses

the setup script. So, explicitly pass --dotnet-versions 8.0.0, so it
can resolve to the available sdk.

Traceback (most recent call last):
  File "/mnt/vss/_work/1/s/Payload/performance/scripts/ci_setup.py", line 357, in <module>
    __main(sys.argv[1:])
  File "/mnt/vss/_work/1/s/Payload/performance/scripts/ci_setup.py", line 310, in __main
    dotnet_version = dotnet.get_dotnet_version(target_framework_moniker, args.cli) if args.dotnet_versions == [] else args.dotnet_versions[0]
  File "/mnt/vss/_work/1/s/Payload/performance/scripts/dotnet.py", line 568, in get_dotnet_version
    "Unable to determine the .NET SDK used for {}".format(framework)
RuntimeError: Unable to determine the .NET SDK used for net7.0

Use test-main.js from 7.0 branch for perf pipeline

Since we use a sdk+workload for running the perf benchmarks, and build
the projects for tfm=net7.0, it uses the 7.0 runtime pack, which has
some differences. So, use the 7.0 test-main.js .

Fails with:

[2022/09/11 01:06:47][INFO] test-main.js:262: TypeError: dotnet.withVirtualWorkingDirectory(...).withEnvironmentVariables(...).withDiagnosticTracing(...).withExitOnUnhandledError is not a function
[2022/09/11 01:06:47][INFO]             .withExitOnUnhandledError()
[2022/09/11 01:06:47][INFO]              ^
[2022/09/11 01:06:47][INFO] TypeError: dotnet.withVirtualWorkingDirectory(...).withEnvironmentVariables(...).withDiagnosticTracing(...).withExitOnUnhandledError is not a function
[2022/09/11 01:06:47][INFO]     at run (test-main.js:262:14)
[2022/09/11 01:06:47][INFO]
[2022/09/11 01:06:47][INFO] 1 pending unhandled Promise rejection(s) detected.

Fixes #75398 .

ghost · 2022-09-11T00:53:46Z

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

null

Author:	radical
Assignees:	-
Labels:	`NO-REVIEW`, `arch-wasm`, `area-Infrastructure-mono`, `perf-pipeline`
Milestone:	-

radical · 2022-09-11T00:53:53Z

/azp run runtime-wasm-perf

azure-pipelines · 2022-09-11T00:54:05Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-09-11T02:15:19Z

/azp run runtime-wasm-perf

azure-pipelines · 2022-09-11T02:15:30Z

Azure Pipelines successfully started running 1 pipeline(s).

Build SizeOnDisk, and Startup tools for `net7.0` instead of `net6.0`. It current fails with: ``` error NU1102: Unable to find package Microsoft.NETCore.App.Ref with version (= 6.0.10) error NU1102: - Found 2073 version(s) in dotnet6 [ Nearest version: 6.0.2-servicing.1.22101.5 ] error NU1102: - Found 1520 version(s) in dotnet7 [ Nearest version: 7.0.0-alpha.1.21417.28 ] error NU1102: - Found 1120 version(s) in dotnet5 [ Nearest version: 6.0.0-alpha.1.20420.3 ] error NU1102: - Found 76 version(s) in dotnet3.1 [ Nearest version: 3.1.1-servicing.19602.11 ] error NU1102: - Found 52 version(s) in dotnet-public [ Nearest version: 7.0.0-preview.1.22076.8 ] error NU1102: - Found 1 version(s) in dotnet-eng [ Nearest version: 5.0.0-alpha.1.19618.1 ] error NU1102: - Found 0 version(s) in benchmark-dotnet-prerelease error NU1102: - Found 0 version(s) in dotnet-tools error NU1102: - Found 0 version(s) in dotnet3.1-transport error NU1102: - Found 0 version(s) in dotnet5-transport ```

We are building with 8.0 sdk now, but for tfm=net7.0 and that confuses the setup script. So, explicitly pass `--dotnet-versions 8.0.0`, so it can resolve to the available sdk. ``` Traceback (most recent call last): File "/mnt/vss/_work/1/s/Payload/performance/scripts/ci_setup.py", line 357, in <module> __main(sys.argv[1:]) File "/mnt/vss/_work/1/s/Payload/performance/scripts/ci_setup.py", line 310, in __main dotnet_version = dotnet.get_dotnet_version(target_framework_moniker, args.cli) if args.dotnet_versions == [] else args.dotnet_versions[0] File "/mnt/vss/_work/1/s/Payload/performance/scripts/dotnet.py", line 568, in get_dotnet_version "Unable to determine the .NET SDK used for {}".format(framework) RuntimeError: Unable to determine the .NET SDK used for net7.0 ```

Since we use a sdk+workload for running the perf benchmarks, and build the projects for tfm=net7.0, it uses the 7.0 runtime pack, which has some differences. So, use the 7.0 test-main.js . Fails with: ``` [2022/09/11 01:06:47][INFO] test-main.js:262: TypeError: dotnet.withVirtualWorkingDirectory(...).withEnvironmentVariables(...).withDiagnosticTracing(...).withExitOnUnhandledError is not a function [2022/09/11 01:06:47][INFO] .withExitOnUnhandledError() [2022/09/11 01:06:47][INFO] ^ [2022/09/11 01:06:47][INFO] TypeError: dotnet.withVirtualWorkingDirectory(...).withEnvironmentVariables(...).withDiagnosticTracing(...).withExitOnUnhandledError is not a function [2022/09/11 01:06:47][INFO] at run (test-main.js:262:14) [2022/09/11 01:06:47][INFO] [2022/09/11 01:06:47][INFO] 1 pending unhandled Promise rejection(s) detected. ```

radical · 2022-09-13T16:02:09Z

this needs an approval. Without this dotnet-runtime-perf is broken on main.

LoopedBard3 · 2022-09-12T17:32:52Z

eng/testing/performance/performance-setup.sh

      echo "  --alpine                       Set for runs on Alpine"
      echo "  --iosmono                      Set for ios Mono/Maui runs"
      echo "  --iosllvmbuild                 Set LLVM for iOS Mono/Maui runs"
      echo "  --mauiversion                  Set the maui version for Mono/Maui runs"
      echo ""
-      exit 0
+      exit 1


Why did we change this to 1?

If you pass an unknown argument to the script then it will show help, and exit. But the build will keep going, even though it is missing crucial setup configuration. With exit 1, the build will stop at that point.

That makes sense, this should be a good change.

LoopedBard3 · 2022-09-12T17:40:15Z

eng/pipelines/coreclr/templates/run-performance-job.yml

@@ -162,7 +163,7 @@ jobs:
      displayName: Performance Setup (Unix)
      condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
      continueOnError: ${{ parameters.continueOnError }}
-    - script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments)
+    - script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments) ${{ parameters.additionalSetupParameters }}


It is mentioned that the additionalSetupParameters is passed to performance-setup, although is being passed to ci_setup here. Along with the changes to performance-setup, it looks like this should be added to the lines above this one.

For run-perfomance-job the comments in perf-wasm-jobs.yml say:
additionalSetupParameters: '--dotnet-versions 8.0.0' # passed to ci_setup.py

For run-scenarios-job the comment says:
additionalSetupParameters: '--dotnetversions 8.0.0' # passed to performance-setup.sh

That matches what the ymls are doing. Please correct me if I'm missing something.

LoopedBard3 · 2022-09-12T17:41:20Z

eng/testing/performance/performance-setup.sh

@@ -354,6 +360,10 @@ if [[ -n "$mono_dotnet" && "$monoaot" == "false" ]]; then
    mv $mono_dotnet $mono_dotnet_path
 fi

+if [[ -n "$dotnet_versions" ]]; then
+    setup_arguments="$setup_arguments --dotnet-versions $dotnet_versions"


Is there a corresponding change to ci_setup in the perf repo to use this setup argument? I didn't see anything already implemented.

https://github.com/dotnet/performance/blob/89e9ed2e5bf061d7bc32ce4c0b4773e117fc6304/scripts/dotnet.py#L868-L876

I see, I missed the dotnet parser args being added to the ci_setup parser.

LoopedBard3

LGTM

radical added NO-REVIEW Experimental/testing PR, do NOT review it arch-wasm WebAssembly architecture area-Infrastructure-mono perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines labels Sep 11, 2022

ghost assigned radical Sep 11, 2022

radical added 2 commits September 11, 2022 04:06

radical force-pushed the fix-perf-pipeline branch from 1daabee to 20e010f Compare September 11, 2022 08:14

radical marked this pull request as ready for review September 11, 2022 20:15

radical requested review from LoopedBard3, adamsitnik, DrewScoggins and lewing September 11, 2022 20:15

radical removed the NO-REVIEW Experimental/testing PR, do NOT review it label Sep 12, 2022

radical mentioned this pull request Sep 12, 2022

[release/7.0][wasm] Fix broken dotnet-runtime-perf #75408

Merged

LoopedBard3 requested changes Sep 13, 2022

View reviewed changes

ghost added needs-author-action An issue or pull request that requires more info or actions from the author. and removed needs-author-action An issue or pull request that requires more info or actions from the author. labels Sep 13, 2022

LoopedBard3 approved these changes Sep 13, 2022

View reviewed changes

radical merged commit 8293726 into dotnet:main Sep 13, 2022

radical deleted the fix-perf-pipeline branch September 13, 2022 17:13

ghost locked as resolved and limited conversation to collaborators Oct 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasm] Fix perf pipeline errors #75406

[wasm] Fix perf pipeline errors #75406

radical commented Sep 11, 2022 •

edited

Loading

ghost commented Sep 11, 2022

radical commented Sep 11, 2022

azure-pipelines bot commented Sep 11, 2022

radical commented Sep 11, 2022

azure-pipelines bot commented Sep 11, 2022

radical commented Sep 13, 2022

LoopedBard3 Sep 12, 2022

radical Sep 13, 2022

LoopedBard3 Sep 13, 2022

LoopedBard3 Sep 12, 2022

radical Sep 13, 2022

LoopedBard3 Sep 12, 2022

radical Sep 13, 2022

LoopedBard3 Sep 13, 2022

LoopedBard3 left a comment

[wasm] Fix perf pipeline errors #75406

[wasm] Fix perf pipeline errors #75406

Conversation

radical commented Sep 11, 2022 • edited Loading

Build SizeOnDisk, and Startup tools for net7.0 instead of net6.0.

We are building with 8.0 sdk now, but for tfm=net7.0 and that confuses

Use test-main.js from 7.0 branch for perf pipeline

ghost commented Sep 11, 2022

radical commented Sep 11, 2022

azure-pipelines bot commented Sep 11, 2022

radical commented Sep 11, 2022

azure-pipelines bot commented Sep 11, 2022

radical commented Sep 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LoopedBard3 left a comment

Choose a reason for hiding this comment

radical commented Sep 11, 2022 •

edited

Loading

Build SizeOnDisk, and Startup tools for `net7.0` instead of `net6.0`.