Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ArPow stage 1: local source-build infrastructure #53294

Merged
merged 50 commits into from
Jun 22, 2021
Merged

ArPow stage 1: local source-build infrastructure #53294

Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
1054536
Initial arcade-powered source-build infra
dagood Feb 4, 2021
7d0c956
Add patches, fixup from 5.0
dagood Feb 4, 2021
376c10c
Rename patches to match 6.0-p1 work
dagood Feb 4, 2021
8a4c258
Add source-build specific build script
dseefeld Jan 15, 2020
9d474ce
Incorporate build-source-build.sh, into eng/
dagood Feb 4, 2021
06f1d5e
Run inner build script, through coreclr
dagood Feb 8, 2021
0e4d155
Initial source-build args based on live build (not props file)
dagood Feb 8, 2021
cc4124f
Cleanup: add RID comments, rm empty/absolute args
dagood Feb 8, 2021
59fb50e
Fix subsets
dagood Feb 8, 2021
ad4201b
Disable sourcelink in env, not args
dagood Feb 8, 2021
1c87b39
Fix CI NZEC
dagood Feb 8, 2021
c4f711d
Revert "Use same code to DetectCiphersuiteConfiguration for portable …
dagood Feb 8, 2021
479c95a
Fix portability
dagood Feb 8, 2021
2565431
Fix AllJits build
dagood Feb 8, 2021
cd313cd
Fix missing crossgen2 for non-portable RID
dagood Feb 12, 2021
2b5307d
Create supplemental intermediate nupkgs
dagood Feb 12, 2021
d7546c5
Tweak category names
dagood Feb 16, 2021
d6602eb
Use centralized supplemental nupkg infra
dagood Feb 16, 2021
843caa7
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Apr 27, 2021
a7d8b95
Add additional 6.0 patches
MichaelSimons Apr 27, 2021
18d9616
Patch updates after merging in main
MichaelSimons Apr 27, 2021
f8e8475
SourceBuild.props cleanup
MichaelSimons Apr 27, 2021
aa13d3c
Fix issue with incorrect patch merge
MichaelSimons May 3, 2021
38cea15
Patch updates
MichaelSimons May 4, 2021
f5445ee
Edit clr.tools patch
MichaelSimons May 14, 2021
4d94fd4
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons May 14, 2021
0a8059d
patch updates
MichaelSimons May 14, 2021
9b2c5b3
Revert patch integration
MichaelSimons May 14, 2021
1b2d9b1
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons May 25, 2021
23e4735
Patch updates
MichaelSimons May 26, 2021
5a12eaf
Edits per code review feedback
MichaelSimons Jun 8, 2021
e0c3357
ILAsm workaround
MichaelSimons Jun 8, 2021
658dea4
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Jun 8, 2021
de586df
patch updates
MichaelSimons Jun 8, 2021
e9b7fe4
Move logic to set ILAsmToolPath for source-build
MichaelSimons Jun 8, 2021
469f6a1
Update eng/SourceBuild.props
MichaelSimons Jun 9, 2021
6c2e477
Remove libraries specific patches
ViktorHofer Jun 14, 2021
1c97495
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Jun 14, 2021
e6f5d8f
Patch updates necessary with latest main merge
MichaelSimons Jun 14, 2021
f026627
Add back libraries-packages.proj harvesting patch
MichaelSimons Jun 14, 2021
371a532
Refactor intermediate package split to be chunkier
MichaelSimons Jun 14, 2021
58bb935
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Jun 15, 2021
309421d
Integrate patch 0017
MichaelSimons Jun 15, 2021
7230fa1
Subsets update per code review
MichaelSimons Jun 15, 2021
972035a
Remove obsolete patch
MichaelSimons Jun 15, 2021
f905b0e
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Jun 15, 2021
0124b4f
Merge branch 'main' of https://github.com/dotnet/runtime into arpow-s…
MichaelSimons Jun 18, 2021
cf48c62
Removed patches that were integrated into main
MichaelSimons Jun 18, 2021
4211c03
Remove two additional patches
ViktorHofer Jun 21, 2021
88a1715
Remove remaining patches
ViktorHofer Jun 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
<WasmBuildTasksAssemblyPath>$([MSBuild]::NormalizePath('$(WasmBuildTasksDir)', 'WasmBuildTasks.dll'))</WasmBuildTasksAssemblyPath>
<MonoAOTCompilerTasksAssemblyPath>$([MSBuild]::NormalizePath('$(MonoAOTCompilerDir)', 'MonoAOTCompiler.dll'))</MonoAOTCompilerTasksAssemblyPath>
<RuntimeConfigParserTasksAssemblyPath>$([MSBuild]::NormalizePath('$(RuntimeConfigParserDir)', 'RuntimeConfigParser.dll'))</RuntimeConfigParserTasksAssemblyPath>
<ILAsmToolPath Condition="'$(DotNetBuildFromSource)' == 'true'">$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'coreclr', '$(TargetOS).$(TargetArchitecture).$(Configuration)'))</ILAsmToolPath>
</PropertyGroup>

<PropertyGroup Label="CalculateConfiguration">
Expand Down
88 changes: 88 additions & 0 deletions eng/SourceBuild.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
<Project>

<PropertyGroup>
<GitHubRepositoryName>runtime</GitHubRepositoryName>
MichaelSimons marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>

<!-- Set up the dotnet/runtime source-build command. -->
<PropertyGroup>
<BaseInnerSourceBuildCommand>./build.sh</BaseInnerSourceBuildCommand>

<SourceBuildPortable>true</SourceBuildPortable>
<SourceBuildPortable Condition="'$(SourceBuildNonPortable)' == 'true'">false</SourceBuildPortable>

<!-- If TargetRid not specified, detect RID based on portability. -->
<TargetRid Condition="'$(TargetRid)' == '' and '$(SourceBuildNonPortable)' == 'true'">$([System.Runtime.InteropServices.RuntimeInformation]::RuntimeIdentifier)</TargetRid>
<TargetRid Condition="'$(TargetRid)' == ''">$(__DistroRid)</TargetRid>

<!-- Split e.g. 'fedora.33-x64' into 'fedora.33' and 'x64'. -->
<_targetRidPlatformIndex>$(TargetRid.LastIndexOfAny("-"))</_targetRidPlatformIndex>
<TargetRidWithoutPlatform>$(TargetRid.Substring(0, $(_targetRidPlatformIndex)))</TargetRidWithoutPlatform>
<TargetRidPlatform>$(TargetRid.Substring($(_targetRidPlatformIndex)).TrimStart('-'))</TargetRidPlatform>

<LogVerbosity Condition="'$(LogVerbosity)' == ''">minimal</LogVerbosity>
</PropertyGroup>

<ItemGroup>
<!-- Work around issue where local clone may cause failure using non-origin remote fallback: https://github.com/dotnet/sourcelink/issues/629 -->
<InnerBuildEnv Include="EnableSourceControlManagerQueries=false" />
<InnerBuildEnv Include="EnableSourceLink=false" />
<InnerBuildEnv Include="DisableSourceLink=true" />
<InnerBuildEnv Include="DeterministicSourcePaths=false" />
</ItemGroup>

<Target Name="GetRuntimeSourceBuildCommandConfiguration"
BeforeTargets="GetSourceBuildCommandConfiguration">
<PropertyGroup>
<InnerBuildArgs>$(InnerBuildArgs) --arch $(TargetRidPlatform)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --configuration $(Configuration)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --ci</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --allconfigurations</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --verbosity $(LogVerbosity)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --nodereuse false</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --warnAsError false</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) --cmakeargs -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=TRUE</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:MicrosoftNetFrameworkReferenceAssembliesVersion=1.0.0</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:ContinuousIntegrationBuild=true</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:PackageRid=$(TargetRid)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:NoPgoOptimize=true</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:KeepNativeSymbols=true</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:RuntimeOS=$(TargetRidWithoutPlatform)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:PortableBuild=$(SourceBuildPortable)</InnerBuildArgs>
<InnerBuildArgs>$(InnerBuildArgs) /p:BuildDebPackage=false</InnerBuildArgs>
</PropertyGroup>
</Target>

<!-- Remove once patches are integrated into the repo -->
<Target Name="ApplySourceBuildPatchFiles"
AfterTargets="PrepareInnerSourceBuildRepoRoot"
BeforeTargets="RunInnerSourceBuildCommand">

<ItemGroup>
<SourceBuildPatchFile Include="$(RepositoryEngineeringDir)source-build-patches\*.patch" />
</ItemGroup>

<Exec
Command="git apply --ignore-whitespace --whitespace=nowarn &quot;%(SourceBuildPatchFile.FullPath)&quot;"
WorkingDirectory="$(InnerSourceBuildRepoRoot)"
Condition="'@(SourceBuildPatchFile)' != ''" />
</Target>

<Target Name="CategorizeRuntimeSupplementalArtifacts"
BeforeTargets="GetCategorizedIntermediateNupkgContents">
<ItemGroup>
<!--
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI - I logged an issue for having the ArPOW infrastructure handle large packages.

Runtime artifacts are too large to fit into a single package (Azure DevOps feeds 500 mb constraint).
Split large components into separate packages.
-->
<IntermediateNupkgArtifactFile Include="$(CurrentRepoSourceBuildArtifactsPackagesDir)Shipping\dotnet-runtime-*.tar.gz" Category="runtime" />

<IntermediateNupkgArtifactFile
Include="
$(CurrentRepoSourceBuildArtifactsPackagesDir)Shipping\dotnet-crossgen2-*.tar.gz;
$(CurrentRepoSourceBuildArtifactsPackagesDir)Shipping\Microsoft.NETCore.App.Crossgen2.*.nupkg"
Category="Crossgen2Pack" />
</ItemGroup>
</Target>

</Project>
5 changes: 5 additions & 0 deletions eng/SourceBuildPrebuiltBaseline.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<UsageData>
<IgnorePatterns>
<UsagePattern IdentityGlob="*/*" />
ViktorHofer marked this conversation as resolved.
Show resolved Hide resolved
</IgnorePatterns>
</UsageData>
21 changes: 12 additions & 9 deletions eng/Subsets.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
<PropertyGroup>
<DefaultSubsets>clr+mono+libs+host+packs</DefaultSubsets>
<DefaultSubsets Condition="'$(TargetsMobile)' == 'true'">mono+libs+packs</DefaultSubsets>
<!-- On Windows mono is supported for x86/x64 only -->
<DefaultSubsets Condition="'$(TargetOS)' == 'windows' and '$(TargetArchitecture)' != 'x86' and '$(TargetArchitecture)' != 'x64'">clr+libs+host+packs</DefaultSubsets>
<!-- mono is not supported in source build. On Windows mono is supported for x86/x64 only. -->
<DefaultSubsets Condition="'$(DotNetBuildFromSource)' == 'true' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' != 'x86' and '$(TargetArchitecture)' != 'x64')">clr+libs+host+packs</DefaultSubsets>
</PropertyGroup>

<!-- Init _subset here in to allow RuntimeFlavor to be set as early as possible -->
Expand All @@ -49,18 +49,21 @@
<DefaultMonoSubsets Condition="'$(TargetOS)' == 'Browser'">$(DefaultMonoSubsets)mono.wasmruntime+</DefaultMonoSubsets>
<DefaultMonoSubsets Condition="'$(MonoCrossAOTTargetOS)' != ''">$(DefaultMonoSubsets)mono.aotcross+</DefaultMonoSubsets>
<DefaultMonoSubsets>$(DefaultMonoSubsets)mono.runtime+mono.corelib+mono.packages</DefaultMonoSubsets>

<DefaultLibrariesSubsets Condition="'$(BuildTargetFramework)' == '$(NetCoreAppCurrent)' or
'$(BuildTargetFramework)' == '' or
'$(BuildAllConfigurations)' == 'true'">libs.native+</DefaultLibrariesSubsets>
<DefaultLibrariesSubsets>$(DefaultLibrariesSubsets)libs.ref+libs.src+libs.pretest+libs.packages</DefaultLibrariesSubsets>
<DefaultLibrariesSubsets>$(DefaultLibrariesSubsets)libs.ref+libs.src+libs.packages</DefaultLibrariesSubsets>
<DefaultLibrariesSubsets Condition="'$(DotNetBuildFromSource)' != 'true'">$(DefaultLibrariesSubsets)+libs.pretest</DefaultLibrariesSubsets>

<DefaultHostSubsets>host.native+host.pkg+host.tools+host.tests</DefaultHostSubsets>
<DefaultHostSubsets>host.native+host.tools</DefaultHostSubsets>
<DefaultHostSubsets Condition="'$(DotNetBuildFromSource)' != 'true'">$(DefaultHostSubsets)+host.pkg+host.tests</DefaultHostSubsets>
<DefaultHostSubsets Condition="'$(RuntimeFlavor)' == 'Mono'"></DefaultHostSubsets>
<DefaultHostSubsets Condition="'$(RuntimeFlavor)' == 'Mono' and '$(TargetsMobile)' != 'true'">host.native</DefaultHostSubsets>

<DefaultPacksSubsets>packs.product</DefaultPacksSubsets>
<DefaultPacksSubsets Condition="'$(BuildMonoAOTCrossCompilerOnly)' != 'true'">$(DefaultPacksSubsets)+packs.tests</DefaultPacksSubsets>
<DefaultPacksSubsets Condition="'$(DotNetBuildFromSource)' == 'true'">$(DefaultPacksSubsets)+packs.installers</DefaultPacksSubsets>
</PropertyGroup>

<PropertyGroup>
Expand Down Expand Up @@ -204,12 +207,12 @@
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\runincontext\runincontext.csproj;
$(CoreClrProjectRoot)tools\r2rdump\R2RDump.csproj;
$(CoreClrProjectRoot)tools\dotnet-pgo\dotnet-pgo.csproj;
$(CoreClrProjectRoot)tools\r2rtest\R2RTest.csproj;
$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2.csproj" Category="clr" />
$(CoreClrProjectRoot)tools\r2rtest\R2RTest.csproj" Category="clr" Condition="'$(DotNetBuildFromSource)' != 'true'"/>
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2.csproj" Category="clr" />

<ProjectToBuild Condition="'$(TargetArchitecture)' != 'x64' and '$(BuildArchitecture)' == 'x64'" Include="$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2_crossarch.csproj" Category="clr" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler.TypeSystem.ReadyToRun.Tests\ILCompiler.TypeSystem.ReadyToRun.Tests.csproj"
Test="true" Category="clr" Condition="'$(__DistroRid)' != 'linux-musl-x64'"/>
Test="true" Category="clr" Condition="'$(__DistroRid)' != 'linux-musl-x64' and '$(DotNetBuildFromSource)' != 'true'"/>
</ItemGroup>

<ItemGroup Condition="$(_subset.Contains('+clr.nativecorelib+'))">
Expand All @@ -221,7 +224,7 @@

<ItemGroup Condition="$(_subset.Contains('+clr.packages+')) and '$(PgoInstrument)' != 'true'">
<ProjectToBuild Include="$(CoreClrProjectRoot).nuget\coreclr-packages.proj" Pack="true" Category="clr" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\dotnet-pgo\dotnet-pgo-pack.proj" Pack="true" BuildInParallel="false" Category="clr" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\dotnet-pgo\dotnet-pgo-pack.proj" Pack="true" BuildInParallel="false" Category="clr" Condition="'$(DotNetBuildFromSource)' != 'true'"/>
</ItemGroup>

<!-- Mono sets -->
Expand Down
1 change: 1 addition & 0 deletions eng/Version.Details.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
<SourceBuild RepoName="arcade" ManagedOnly="true" />
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
Expand Down
4 changes: 0 additions & 4 deletions eng/packaging.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@
</PropertyGroup>

<PropertyGroup Condition="'$(DotNetBuildFromSource)' == 'true'">
<!-- Don't build referenced projects -->
<BuildPackageLibraryReferences>false</BuildPackageLibraryReferences>
<!-- Omit any files that were not built -->
<AllowPartialPackages>true</AllowPartialPackages>
<!-- Don't permit harvesting since this requires pre-builts -->
<HarvestStablePackage>false</HarvestStablePackage>
<!-- Validation will fail in case we were relying on harvested assets or assets not built to satisfy stated support -->
Expand Down
25 changes: 25 additions & 0 deletions eng/source-build-patches/0019-Fix-missing-crossgen2-for-non-portable-RID.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From a3b356e19adaa1b001fdf6de79896ed85ec74df5 Mon Sep 17 00:00:00 2001
From: Davis Goodin <dagood@microsoft.com>
Date: Fri, 12 Feb 2021 11:11:04 -0600
Subject: [PATCH] Fix missing crossgen2 for non-portable RID

---
.../Microsoft.NETCore.App.Crossgen2.sfxproj | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj b/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj
index ec198224a2e..51dc61d8783 100644
--- a/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj
+++ b/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj
@@ -9,7 +9,8 @@
<OverridePackageId>$(SharedFrameworkName)$(PgoSuffix).$(RuntimeIdentifier)</OverridePackageId>
<ArchiveName>dotnet-crossgen2</ArchiveName>
<SharedFrameworkHostFileNameOverride>crossgen2</SharedFrameworkHostFileNameOverride>
- <RuntimeIdentifiers>linux-x64;linux-musl-x64;linux-arm;linux-musl-arm;linux-arm64;linux-musl-arm64;osx-x64;osx-arm64;win-x64;win-x86;win-arm64;win-arm</RuntimeIdentifiers>
+ <!-- Build this pack for any RID if building from source. Otherwise, only build a few RIDs. -->
ViktorHofer marked this conversation as resolved.
Show resolved Hide resolved
+ <RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' != 'true'">linux-x64;linux-musl-x64;linux-arm;linux-musl-arm;linux-arm64;linux-musl-arm64;osx-x64;osx-arm64;win-x64;win-x86;win-arm64;win-arm</RuntimeIdentifiers>
<GenerateInstallers>false</GenerateInstallers>
<GetSharedFrameworkFilesForReadyToRunDependsOn>
AddRuntimeFilesToPackage;
--
2.25.4
25 changes: 25 additions & 0 deletions eng/source-build-patches/0029-libraries-packages-use-net5.0-PackagingTaskA.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From da5a2c2593df373d1fafa605e1d7ce3ba2ca38e7 Mon Sep 17 00:00:00 2001
From: Tom Deseyn <tom.deseyn@gmail.com>
Date: Wed, 18 Nov 2020 09:29:17 +0100
Subject: [PATCH] runtime: libraries-packages: use net5.0 PackagingTaskAssembly

---
src/libraries/libraries-packages.proj | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/libraries/libraries-packages.proj b/src/libraries/libraries-packages.proj
index 603d523df55..38cd7e74337 100644
--- a/src/libraries/libraries-packages.proj
+++ b/src/libraries/libraries-packages.proj
@@ -7,7 +7,7 @@

<PropertyGroup>
<PackagingTaskAssembly>$(NuGetPackageRoot)microsoft.dotnet.build.tasks.packaging\$(MicrosoftDotNetBuildTasksPackagingVersion)\tools\</PackagingTaskAssembly>
- <PackagingTaskAssembly Condition="'$(MSBuildRuntimeType)' == 'core'">$(PackagingTaskAssembly)netcoreapp3.1\</PackagingTaskAssembly>
+ <PackagingTaskAssembly Condition="'$(MSBuildRuntimeType)' == 'core'">$(PackagingTaskAssembly)net5.0\</PackagingTaskAssembly>
<PackagingTaskAssembly Condition="'$(MSBuildRuntimeType)' != 'core'">$(PackagingTaskAssembly)net472\</PackagingTaskAssembly>
<PackagingTaskAssembly>$(PackagingTaskAssembly)Microsoft.DotNet.Build.Tasks.Packaging.dll</PackagingTaskAssembly>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This usage can be deleted if this project just uses the package's props to define the task: https://github.com/dotnet/arcade/blob/13bb7c843fac9ce3652b5f418ca27b9bf3b83754/src/Microsoft.DotNet.Build.Tasks.Packaging/src/build/Packaging.common.targets#L44

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently necessary because of dotnet/arcade#7413

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, should be simple enough to write those targets with a template task? At the very least, why isn't this patch in Arcade instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the very least, why isn't this patch in Arcade instead?

What would you patch in Arcade? For whatever reason, the Microsoft.DotNet.Build.Tasks.Packaging package doesn't expose a property that points to the msbuild task assembly.

I wouldn't want to invest any resources in fixing this aside from inlining the patch into main as this code-path will go away when @Anipik finished the pkgproj migration.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This usage can be deleted if this project just uses the package's props to define the task

The project can't use the package's props or targets files as it contributes to the project's restore. This runs as part of restore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you patch in Arcade

My impression based on your comment This is currently necessary because of dotnet/arcade#7413 was that the props/targets didn't have the right path.

If the props have the right path, then this proj file can just import the props/targets. Since all this proj does is define a task which those targets already define.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The project can't use the package's props or targets files as it contributes to the project's restore.

It should be downloaded before that, I think, as part of

runtime/eng/Tools.props

Line 9 in 9199eb6

<PackageReference Include="Microsoft.DotNet.Build.Tasks.Packaging" Version="$(MicrosoftDotNetBuildTasksPackagingVersion)" />
.

I'd prefer to make things less hacky if possible though, was hoping your conversion of this to a traversal project would just let it naturally consume the package and let us delete stuff. If that's not the case then I'm fine leaving this since it'll go away with PkgProj removal, as you said.

Copy link
Member

@ViktorHofer ViktorHofer Jun 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be downloaded before that, I think, as part of

Right, that's how the package is available before the project's restore to calculate the PackageDownload items. Libraries-packages.proj itself would need to manually import the props and targets files which IMO isn't great either. Let's just keep it as-is for now and apply the patch.

</PropertyGroup>
--
2.25.4

25 changes: 25 additions & 0 deletions eng/source-build-patches/0034-Exclude-Microsoft.Extensions.Hosting.WindowsServices.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From 6521f8a43f5c146efc72b72dd3b0e61344a71618 Mon Sep 17 00:00:00 2001
From: dseefeld <dseefeld@microsoft.com>
Date: Mon, 15 Mar 2021 19:00:55 +0000
Subject: [PATCH] Exclude Microsoft.Extensions.Hosting.WindowsServices from
source-build

---
.../Directory.Build.props | 3 ++
1 file changed, 3 insertions(+)

diff --git a/src/libraries/Microsoft.Extensions.Hosting.WindowsServices/Directory.Build.props b/src/libraries/Microsoft.Extensions.Hosting.WindowsServices/Directory.Build.props
index c5f4d2e6da1..5493f19bc19 100644
--- a/src/libraries/Microsoft.Extensions.Hosting.WindowsServices/Directory.Build.props
+++ b/src/libraries/Microsoft.Extensions.Hosting.WindowsServices/Directory.Build.props
@@ -2,5 +2,8 @@
<PropertyGroup>
<PackageTags>hosting</PackageTags>
<SupportedOSPlatforms>windows</SupportedOSPlatforms>
+ <!-- Source build does not support Windows. -->
+ <ExcludeFromSourceBuild>true</ExcludeFromSourceBuild>
+ <IsPackable>false</IsPackable>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dseefeld why is this necessary? Unfortunately the comment or the commit message doesn't provide any details.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that since the project was only supported on Windows it wasn't necessary for source-build.

Copy link
Member

@ericstj ericstj Jun 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it hurt anything to build it? We have plenty of other things that are the same and they don't get excluded from source build. https://github.com/dotnet/runtime/tree/main/src/libraries/System.ServiceProcess.ServiceController for example.
I'd recommend just deleting this patch if all it is doing is producing one less package.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than likely it was introducing prebuilts. Until we get the prebuilt report cleaned up during this ArPow effort it is difficult to detect. @dseefeld, are you alright with removing the patch until we can prove it is necessary?

</PropertyGroup>
</Project>
--
2.25.4
27 changes: 27 additions & 0 deletions eng/source-build-patches/0035-cross-gen-corelib-no-mibc-files.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
From b473825eaa9540131b9b7181274e3e6547eed02b Mon Sep 17 00:00:00 2001
From: Michael Simons <msimons@microsoft.com>
Date: Mon, 3 May 2021 15:29:02 +0000
Subject: [PATCH] cross-gen-corelib.proj fix related to PR 50536 to handle no
mibc files

---
src/coreclr/crossgen-corelib.proj | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/coreclr/crossgen-corelib.proj b/src/coreclr/crossgen-corelib.proj
index 53c4e57..0a8f17a 100644
--- a/src/coreclr/crossgen-corelib.proj
+++ b/src/coreclr/crossgen-corelib.proj
@@ -100,7 +100,8 @@
<CrossGenDllCmd>$(CrossGenDllCmd) -o:$(CoreLibOutputPath)</CrossGenDllCmd>
<CrossGenDllCmd>$(CrossGenDllCmd) -r:$([MSBuild]::NormalizePath('$(BinDir)', 'IL', '*.dll'))</CrossGenDllCmd>
<CrossGenDllCmd>$(CrossGenDllCmd) --targetarch:$(TargetArchitecture)</CrossGenDllCmd>
- <CrossGenDllCmd Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true'">$(CrossGenDllCmd) -m:$(MergedMibcPath) --embed-pgo-data</CrossGenDllCmd>
+ <MibcArgs>@(OptimizationMibcFiles->'-m:$(MergedMibcPath)', ' ')</MibcArgs>
hoyosjs marked this conversation as resolved.
Show resolved Hide resolved
+ <CrossGenDllCmd Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true'">$(CrossGenDllCmd) $(MibcArgs) --embed-pgo-data</CrossGenDllCmd>
<CrossGenDllCmd>$(CrossGenDllCmd) -O</CrossGenDllCmd>
<CrossGenDllCmd Condition="'$(Configuration)' == 'Debug' or '$(Configuration)' == 'Checked'">$(CrossGenDllCmd) --verify-type-and-field-layout</CrossGenDllCmd>
<CrossGenDllCmd>$(CrossGenDllCmd) $(CoreLibInputPath)</CrossGenDllCmd>
--
1.8.3.1

27 changes: 27 additions & 0 deletions eng/source-build-patches/0036-Exclude-ReadyToRun.Experimental-NuGet-package.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
From 39e09e379d1511bdf4d9484f1ae0efdd516894fa Mon Sep 17 00:00:00 2001
From: Michael Simons <msimons@microsoft.com>
Date: Mon, 3 May 2021 15:35:38 +0000
Subject: [PATCH] Exclude ReadyToRun.Experimental NuGet package

Not required by SDK and depends on non-SB components (e.g. r2rdump)
---
src/coreclr/.nuget/coreclr-packages.proj | 3 +++
1 file changed, 3 insertions(+)

diff --git a/src/coreclr/.nuget/coreclr-packages.proj b/src/coreclr/.nuget/coreclr-packages.proj
index 2f7d06a..b8aec3a 100644
--- a/src/coreclr/.nuget/coreclr-packages.proj
+++ b/src/coreclr/.nuget/coreclr-packages.proj
@@ -13,6 +13,9 @@
<ProjectReference Include="Microsoft.NETCore.TestHost\Microsoft.NETCore.TestHost.proj" />
<ProjectReference Include="Microsoft.NETCore.ILAsm\Microsoft.NETCore.ILAsm.proj" />
<ProjectReference Include="Microsoft.NETCore.ILDAsm\Microsoft.NETCore.ILDAsm.proj" />
+ </ItemGroup>
+
+ <ItemGroup Condition="'$(DotNetBuildFromSource)' != 'true'">
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integrating this patch with #54222.

<ProjectReference Include="ILCompiler.Reflection.ReadyToRun.Experimental\ILCompiler.Reflection.ReadyToRun.Experimental.pkgproj" />
</ItemGroup>

--
1.8.3.1

1 change: 0 additions & 1 deletion src/libraries/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
<PropertyGroup>
<!-- Build all .NET Framework configurations when net48 is passed in. This is for convenience. -->
<AdditionalBuildTargetFrameworks Condition="'$(BuildTargetFramework)' == 'net48'">net461;net462;net47;net471;net472</AdditionalBuildTargetFrameworks>
<AdditionalBuildTargetFrameworks Condition="'$(DotNetBuildFromSource)' == 'true' and '$(BuildAllProjects)' == 'true'">$(AdditionalBuildTargetFrameworks);netstandard2.0</AdditionalBuildTargetFrameworks>
</PropertyGroup>

<!-- Define test projects and companions -->
Expand Down
6 changes: 1 addition & 5 deletions src/libraries/libraries-packages.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
<Project Sdk="Microsoft.Build.Traversal" DefaultTargets="Pack">

<PropertyGroup>
<AdditionalBuildTargetFrameworks Condition="'$(DotNetBuildFromSource)' == 'true'">$(AdditionalBuildTargetFrameworks);package-$(Configuration)</AdditionalBuildTargetFrameworks>
</PropertyGroup>

<PropertyGroup>
<PackagingTaskAssembly>$(NuGetPackageRoot)microsoft.dotnet.build.tasks.packaging\$(MicrosoftDotNetBuildTasksPackagingVersion)\tools\</PackagingTaskAssembly>
<PackagingTaskAssembly Condition="'$(MSBuildRuntimeType)' == 'core'">$(PackagingTaskAssembly)netcoreapp3.1\</PackagingTaskAssembly>
Expand All @@ -14,7 +10,7 @@
<ItemGroup>
<ProjectReference Include="$(PkgDir)*\*.proj" Exclude="$(PkgDir)test\*" />
<ProjectReference Include="$(MSBuildThisFileDirectory)*\src\*.*proj"
Condition="('$(BuildAllConfigurations)' == 'true' or '$(DotNetBuildFromSource)' == 'true')"
Condition="'$(BuildAllConfigurations)' == 'true'"
Exclude="$(MSBuildThisFileDirectory)*\src\**\*.shproj"/>
</ItemGroup>

Expand Down