Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm] Integrate naot-llvm into workload manifest #101801

Closed
wants to merge 35 commits into from
Closed
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
9f4447b
Disable mono workload. Override ILCompiler packs. Add restore feed
maraf May 2, 2024
5904423
Extract the version to a property
maraf May 3, 2024
c7be3fa
Don't override TargetingPackVersion
maraf May 3, 2024
051deab
Add OverrideImportRuntimeIlcPackageTarget
maraf May 7, 2024
f260eea
Add WASI WBT
maraf May 7, 2024
88be64a
Update package version
maraf May 9, 2024
08a98d9
Run with wasmtime
maraf May 9, 2024
d8f8bed
Wasmtime normalizes exit code. Returning 0 seems like a good thing to do
maraf May 9, 2024
8ec46fa
Run the wasi test on CI
maraf May 9, 2024
3a8808a
Explicitly disable browser workload to prevent enabling it in emscrip…
maraf May 9, 2024
30824f6
Browser WBT
maraf May 9, 2024
7946571
Merge remote-tracking branch 'upstream/main' into WasmNaotLlvmSdk
maraf May 9, 2024
4c3c351
Run browser WBT on CI
maraf May 9, 2024
b51f621
Run browser WBT on CI (fix damage)
maraf May 9, 2024
7efc56b
Linux is not supported
maraf May 10, 2024
7bfead3
Run browser WBT on CI (fix class name)
maraf May 10, 2024
a13f2a0
Linux is not supported
maraf May 10, 2024
065a103
Expect fail on non-Windows
maraf May 13, 2024
9b080bb
DEBUG missing libLLVM.dll
maraf May 13, 2024
0096144
DEBUG package reference style project
maraf May 18, 2024
c72e9f4
DEBUG package reference style project (fix)
maraf May 18, 2024
93e54b0
DEBUG package reference style project (fix trimmer)
maraf May 21, 2024
255ec70
DEBUG: OS info
maraf May 22, 2024
b8fb22a
DEBUG: CoreCLR tracing
maraf May 22, 2024
2aa832f
Update NativeAOT-LLVM version
maraf Jun 3, 2024
4e94ebd
Merge branch 'main' into WasmNaotLlvmSdk
maraf Jun 3, 2024
5bd7bf2
Fix merge damage
maraf Jun 4, 2024
4ac36fd
Update LLVM ILC version
maraf Jun 21, 2024
5091f1b
Cleanup after CI debugging
maraf Jun 21, 2024
5c684e6
Merge branch 'main' into WasmNaotLlvmSdk
maraf Jun 21, 2024
9841eb0
Merge branch 'main' into WasmNaotLlvmSdk
maraf Jun 21, 2024
b797fad
Feedback
maraf Aug 14, 2024
1a21a15
Update version
maraf Aug 14, 2024
2c4c818
Run only NativeAOT WBT
maraf Aug 14, 2024
28ebf99
Merge remote-tracking branch 'upstream/main' into WasmNaotLlvmSdk
maraf Aug 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -261,5 +261,6 @@
<EmsdkPackageVersion>$(MicrosoftNETRuntimeEmscriptenVersion)</EmsdkPackageVersion>
<NodePackageVersion>$(runtimewinx64MicrosoftNETCoreRuntimeWasmNodeTransportPackageVersion)</NodePackageVersion>
<EmsdkVersion>3.1.34</EmsdkVersion>
<WasmExperimentalNativeAOTLLVMILCompilerVersion>9.0.0-preview.5.24318.1</WasmExperimentalNativeAOTLLVMILCompilerVersion>
</PropertyGroup>
</Project>
1 change: 1 addition & 0 deletions eng/testing/scenarios/BuildWasiAppsJobsList.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ Wasi.Build.Tests.RuntimeConfigTests
Wasi.Build.Tests.WasiTemplateTests
Wasi.Build.Tests.PInvokeTableGeneratorTests
Wasi.Build.Tests.WasiLibraryModeTests
Wasi.Build.Tests.NativeAOTTests
1 change: 1 addition & 0 deletions eng/testing/scenarios/BuildWasmAppsJobsList.txt
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,4 @@ Wasm.Build.Tests.WorkloadTests
Wasm.Build.Tests.MT.Blazor.SimpleMultiThreadedTests
Wasm.Build.Tests.TestAppScenarios.WasmSdkDebugLevelTests
Wasm.Build.Tests.TestAppScenarios.WasmAppBuilderDebugLevelTests
Wasm.Build.Tests.NativeAOTTests
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
<_WorkloadManifestValues Include="PackageVersionNet6" Value="$(PackageVersionNet6)" />
<_WorkloadManifestValues Include="NetCoreAppCurrent" Value="$(NetCoreAppCurrent)" />
<_WorkloadManifestValues Include="EmscriptenVersion" Value="$(MicrosoftNETRuntimeEmscriptenVersion)" />
<_WorkloadManifestValues Include="WasmExperimentalNativeAOTLLVMILCompilerVersion" Value="$(WasmExperimentalNativeAOTLLVMILCompilerVersion)" />
</ItemGroup>

<GenerateFileFromTemplate
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
<Project>
<PropertyGroup>
<_IsUsingNativeAOT Condition="'$(PublishAot)' == 'true'">true</_IsUsingNativeAOT>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not use PublishAot == true condition directly? MSBuild evaluates it the same way (since there is no use in any target), so it seems redundant.

Copy link
Member Author

@maraf maraf Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can. I wanted to extract it, because we weren't sure if using PublishAot for this highly experimental thing is the right choice.

<_RuntimePackInWorkloadVersionCurrent>${PackageVersion}</_RuntimePackInWorkloadVersionCurrent>
<_RuntimePackInWorkloadVersion8>${PackageVersionNet8}</_RuntimePackInWorkloadVersion8>
<_RuntimePackInWorkloadVersion7>${PackageVersionNet7}</_RuntimePackInWorkloadVersion7>
Expand Down Expand Up @@ -65,6 +66,26 @@
<UsingBrowserRuntimeWorkload Condition="'$(UsingBrowserRuntimeWorkload)' == ''">$(WasmNativeWorkloadAvailable)</UsingBrowserRuntimeWorkload>
</PropertyGroup>

<PropertyGroup Condition="'$(_IsUsingNativeAOT)' == 'true'">
<UseAppHost>false</UseAppHost>
<UsingBrowserRuntimeWorkload Condition="'$(RuntimeIdentifier)' == 'browser-wasm'">false</UsingBrowserRuntimeWorkload>
<UsingWasiRuntimeWorkload Condition="'$(RuntimeIdentifier)' == 'wasi-wasm'">false</UsingWasiRuntimeWorkload>
<BrowserWorkloadDisabled>true</BrowserWorkloadDisabled>
<OverrideImportRuntimeIlcPackageTarget>false</OverrideImportRuntimeIlcPackageTarget>
<WasmExperimentalNativeAOTLLVMILCompilerVersion Condition="'$(WasmExperimentalNativeAOTLLVMILCompilerVersion)' == ''">${WasmExperimentalNativeAOTLLVMILCompilerVersion}</WasmExperimentalNativeAOTLLVMILCompilerVersion>
<RestoreAdditionalProjectSources>$(RestoreAdditionalProjectSources);https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json</RestoreAdditionalProjectSources>
maraf marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>

<ItemGroup Condition="'$(_IsUsingNativeAOT)' == 'true' and ('$(RuntimeIdentifier)' == 'browser-wasm' or '$(RuntimeIdentifier)' == 'wasi-wasm')">
<KnownILCompilerPack Remove="Microsoft.DotNet.ILCompiler" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is me being nit-picky - I would move this down near where we set KnownRuntimePack and the like.

<KnownILCompilerPack Include="Microsoft.DotNet.ILCompiler.LLVM"
TargetFramework="${NetCoreAppCurrent}"
ILCompilerPackNamePattern="runtime.**RID**.Microsoft.DotNet.ILCompiler.LLVM"
ILCompilerPackVersion="$(WasmExperimentalNativeAOTLLVMILCompilerVersion)"
ILCompilerRuntimeIdentifiers="wasi-wasm;browser-wasm;win-x64"
/>
</ItemGroup>

<!-- Mono AOT library mode support -->
<PropertyGroup Condition="'$(PublishAot)' != 'true' and '$(NativeLib)' != ''">
maraf marked this conversation as resolved.
Show resolved Hide resolved
<_IsAndroidLibraryMode Condition="'$(RuntimeIdentifier)' == 'android-arm64' or '$(RuntimeIdentifier)' == 'android-arm' or '$(RuntimeIdentifier)' == 'android-x64' or '$(RuntimeIdentifier)' == 'android-x86'">true</_IsAndroidLibraryMode>
Expand Down Expand Up @@ -186,7 +207,7 @@
<!-- Overrides for wasm threading support -->
<RuntimePackNamePatterns Condition="'$(RuntimeIdentifier)' == 'browser-wasm' and '$(WasmEnableThreads)' == 'true'">Microsoft.NETCore.App.Runtime.Mono.multithread.**RID**</RuntimePackNamePatterns>
</KnownRuntimePack>
<KnownFrameworkReference Update="Microsoft.NETCore.App">
<KnownFrameworkReference Update="Microsoft.NETCore.App" Condition="'$(_IsUsingNativeAOT)' != 'true'">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should override it to the naot version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That version doesn't exit. The naot-llvm produces only ILC-LLVM packages

<TargetingPackVersion Condition="'$(TargetsCurrent)' == 'true'">$(_MonoWorkloadRuntimePackPackageVersion)</TargetingPackVersion>
</KnownFrameworkReference>
<KnownWebAssemblySdkPack Update="@(KnownWebAssemblySdkPack)">
Expand Down
83 changes: 83 additions & 0 deletions src/mono/wasi/Wasi.Build.Tests/NativeAOTTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using Wasm.Build.Tests;
using Xunit;
using Xunit.Abstractions;
using Xunit.Sdk;

#nullable enable

namespace Wasi.Build.Tests;

public class NativeAOTTests : BuildTestBase
{
public NativeAOTTests(ITestOutputHelper output, SharedBuildPerTestClassFixture buildContext)
: base(output, buildContext)
{
}

[Fact]
public void PublishAndRun()
{
const string config = "Release";
string id = $"nativeaot_{GetRandomId()}";
string projectFile = CreateWasmTemplateProject(id, "wasiconsole");
string projectName = Path.GetFileNameWithoutExtension(projectFile);

string programCsContent = File.ReadAllText(Path.Combine(BuildEnvironment.TestAssetsPath, "SimpleMainWithArgs.cs"));
programCsContent = programCsContent.Replace("return 42;", "return 0;");
File.WriteAllText(Path.Combine(_projectDir!, "Program.cs"), programCsContent);
File.Delete(Path.Combine(_projectDir!, "runtimeconfig.template.json"));

var buildArgs = ExpandBuildArgs(new BuildArgs(projectName, config, AOT: false, id, null));

AddItemsPropertiesToProject(projectFile, extraProperties: "<PublishAot>true</PublishAot>");

try
{
bool isWindowsPlatform = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
(_, string buildOutput) = BuildProject(
buildArgs,
id: id,
new BuildProjectOptions(
AssertAppBundle: false,
CreateProject: false,
Publish: true,
TargetFramework: DefaultTargetFramework,
ExpectSuccess: isWindowsPlatform
)
);

if (isWindowsPlatform)
{
string outputDir = Path.Combine(_projectDir!, "bin", config, DefaultTargetFramework, BuildEnvironment.DefaultRuntimeIdentifier, "native");
string outputFileName = $"{id}.wasm";

Assert.True(File.Exists(Path.Combine(outputDir, outputFileName)), $"Expected {outputFileName} to exist in {outputDir}");
Assert.Contains("Generating native code", buildOutput);

using var runCommand = new ToolCommand(BuildEnvironment.GetExecutableName(@"wasmtime"), _testOutput)
.WithWorkingDirectory(outputDir);

var result = runCommand.Execute(outputFileName).EnsureSuccessful();
Assert.Contains("Hello, Wasi Console!", result.Output);
}
else
{
Assert.Contains("NETSDK1204", buildOutput); // Ahead-of-time compilation is not supported on the current platform 'linux-x64'
}
}
finally
{
_testOutput.WriteLine($"Content of {_nugetPackagesDir}");
foreach (string file in Directory.EnumerateFiles(_nugetPackagesDir, "*", SearchOption.AllDirectories))
_testOutput.WriteLine(file);
}
}
}
5 changes: 4 additions & 1 deletion src/mono/wasm/Wasm.Build.Tests/BuildTestBase.cs
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,10 @@ public BuildTestBase(ProjectProviderBase providerBase, ITestOutputHelper output,
var cmd = new DotNetCommand(s_buildEnv, _testOutput)
.WithWorkingDirectory(_projectDir!)
.WithEnvironmentVariable("NUGET_PACKAGES", _nugetPackagesDir)
.WithEnvironmentVariables(buildProjectOptions.ExtraBuildEnvironmentVariables);
.WithEnvironmentVariables(buildProjectOptions.ExtraBuildEnvironmentVariables)
.WithEnvironmentVariable("COREHOST_TRACE", "1")
.WithEnvironmentVariable("COREHOST_TRACEFILE", Environment.GetEnvironmentVariable("HELIX_WORKITEM_UPLOAD_ROOT") ?? string.Empty)
.WithEnvironmentVariable("COREHOST_TRACE_VERBOSITY", "4");
if (UseWBTOverridePackTargets && s_buildEnv.IsWorkload)
cmd.WithEnvironmentVariable("WBTOverrideRuntimePack", "true");

Expand Down
12 changes: 9 additions & 3 deletions src/mono/wasm/Wasm.Build.Tests/Common/BuildEnvironment.cs
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,7 @@ public BuildEnvironment()
EnvVars["WasmEnableWebCil"] = "false";
}

DotNet = Path.Combine(sdkForWorkloadPath!, "dotnet");
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
DotNet += ".exe";
DotNet = GetExecutableName(Path.Combine(sdkForWorkloadPath!, "dotnet"));

if (!string.IsNullOrEmpty(EnvironmentVariables.TestLogPath))
{
Expand Down Expand Up @@ -179,5 +177,13 @@ public bool IsMultiThreadingRuntimePackAvailableFor(string tfm)

protected static string s_directoryBuildPropsForLocal = File.ReadAllText(Path.Combine(TestDataPath, "Local.Directory.Build.props"));
protected static string s_directoryBuildTargetsForLocal = File.ReadAllText(Path.Combine(TestDataPath, "Local.Directory.Build.targets"));

public static string GetExecutableName(string nameOrPath)
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
nameOrPath += ".exe";

return nameOrPath;
}
}
}
80 changes: 80 additions & 0 deletions src/mono/wasm/Wasm.Build.Tests/Templates/NativeAOTTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections.Generic;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using Wasm.Build.Tests.Blazor;
using Xunit;
using Xunit.Abstractions;
using Xunit.Sdk;

#nullable enable

namespace Wasm.Build.Tests;

public class NativeAOTTests : BlazorWasmTestBase
{
public NativeAOTTests(ITestOutputHelper output, SharedBuildPerTestClassFixture buildContext)
: base(output, buildContext)
{
}

[Theory]
[InlineData("NativeAOT")]
public async Task PublishAndRun(string assetName)
{
string config = "Release";
string id = $"browser_{config}_{GetRandomId()}";

InitBlazorWasmProjectDir(id);
Utils.DirectoryCopy(Path.Combine(BuildEnvironment.TestAssetsPath, assetName), Path.Combine(_projectDir!));
string projectName = Path.GetFileNameWithoutExtension(_projectDir!);

try
{
bool isWindowsPlatform = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
(_, string buildOutput) = BuildTemplateProject(
ExpandBuildArgs(new BuildArgs(projectName, config, AOT: false, id, null)),
id: id,
new BuildProjectOptions(
AssertAppBundle: false,
CreateProject: false,
Publish: true,
TargetFramework: DefaultTargetFramework,
ExpectSuccess: isWindowsPlatform
)
);

if (isWindowsPlatform)
{
string outputDir = Path.Combine(_projectDir!, "bin", config, DefaultTargetFramework, BuildEnvironment.DefaultRuntimeIdentifier, "native");

List<string> consoleOutput = new();
BlazorRunOptions blazorRunOptions = new(
CheckCounter: false,
Config: config,
OnConsoleMessage: (_, message) => consoleOutput.Add(message.Text),
Host: BlazorRunHost.WebServer
);
await BlazorRunTest($"{s_xharnessRunnerCommand} wasm webserver --app=. --web-server-use-default-files", outputDir, blazorRunOptions);

Assert.True(consoleOutput.Contains("Hello, NativeAOT!"), $"Expected 'Hello, NativeAOT!' wasn't emitted by the test app. Output was: {String.Join(", ", consoleOutput)}");
}
else
{
Assert.Contains("NETSDK1204", buildOutput); // Ahead-of-time compilation is not supported on the current platform 'linux-x64'
}
}
finally
{
_testOutput.WriteLine($"Is64BitProcess '{Environment.Is64BitProcess}', Is64BitOperatingSystem '{Environment.Is64BitOperatingSystem}', OSVersion '{Environment.OSVersion}'");
_testOutput.WriteLine($"Content of {_nugetPackagesDir}");
foreach (string file in Directory.EnumerateFiles(_nugetPackagesDir, "*", SearchOption.AllDirectories))
_testOutput.WriteLine(file);
}
}
}
13 changes: 13 additions & 0 deletions src/mono/wasm/testassets/NativeAOT/NativeAOT.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<RuntimeIdentifier>browser-wasm</RuntimeIdentifier>
<OutputType>Exe</OutputType>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<DotNetJsApi>true</DotNetJsApi>
<PublishAot>true</PublishAot>
</PropertyGroup>
<ItemGroup>
<WasmExtraFilesToDeploy Include="wwwroot\*" />
</ItemGroup>
</Project>
10 changes: 10 additions & 0 deletions src/mono/wasm/testassets/NativeAOT/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.JavaScript;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

Console.WriteLine("Hello, NativeAOT!");
Console.WriteLine($"Args {String.Join(", ", args)}");
8 changes: 8 additions & 0 deletions src/mono/wasm/testassets/NativeAOT/wwwroot/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<!doctype html>
<html lang="en-us">
<head>
<script type='module' src="./main.js"></script>
</head>
<body>
</body>
</html>
11 changes: 11 additions & 0 deletions src/mono/wasm/testassets/NativeAOT/wwwroot/main.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

import { dotnet } from './dotnet.js'

await dotnet
.withApplicationArguments("A", "B", "C")
.withMainAssembly("NativeAOT")
.create();

await dotnet.run();
Loading