source build prebuilts

[chcosta]

Addresses dotnet/source-build#970 (comment)

Adds a daily Arcade subscription to consume MSBuild updates (NuGet dependencies are not yet available via dependency flow).

@safern is confirming that moving the TargetFramework for Microsoft.DotNet.CodeAnalysis to netstandard2.0 is now safe for CoreFx

[tmat]

This needs corresponding update in the props file that references the task binaries.

[chcosta]

@safern , any update on validating that moving the TargetFramework for Microsoft.DotNet.CodeAnalysis to netstandard2.0 is now safe for CoreFx

[chcosta]

@danmosemsft , this PR is blocked on validating that CoreFx is not broken if we move to netstandard2.0. Do you have any resources that can help us make progress?

[danmoseley]

@ericstj thoughts?

[ericstj]

TFM change is fine but I’m more concerned with the version changes. Those won’t work in the numerous repos that are still building with the 2.x CLI due to MSBuild pinning these in its deps file causing tasks to be downgraded.

Have you tested downstream consumption of these changes?

[ericstj]

Here are a couple examples of regressions this will cause:
#2189
#2277

I also expect there are more here, since nearly everything you are upgrading is in MSBuild's deps file. I suspect you first need to make sure all arcade consumers are on 3.0 CLI before making this change.

[safern]

The problem @chcosta wanted us to review, is that when we targeted the analyzers against .netstandard2.0 we where hitting this when building on VS:

C:\Users\Svick.nuget\packages\microsoft.dotnet.genfacades\1.0.0-beta.18478.5\build\Microsoft.DotNet.GenFacades.targets 97
Error IDE1003 Analyzer assembly 'C:\Users\Svick.nuget\packages\microsoft.dotnet.codeanalysis\1.0.0-beta.18519.7\analyzers\Microsoft.DotNet.CodeAnalysis.dll' depends on 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' but it was not found. Analyzers may not run correctly unless the missing assembly is added as an analyzer reference as well. System.Runtime 1 Active

https://github.com/dotnet/corefx/issues/32980

So that is why I rolled it back to netstandard1.3 following what roslyn does.

[ericstj]

That would only be a problem if we were still running on desktop version or roslyn without netstandard inbox, or running on netcoreapp1.x. We're now running on a roslyn version that targets ns2.0.

I'm more worried about the fallout of these version changes. I spent last week cleaning up regressions that resulted from folks blindly changing these in the past. MSBuild is very brittle here.

[safern]

I'm more worried about the fallout of these version changes. I spent last week cleaning up regressions that resulted from folks blindly changing these in the past. MSBuild is very brittle here.

I agree, that could cause us spend a lot of time investigating and fixing issues like we did with NuGet in the past weeks.

[chcosta]

Thanks, @ericstj, I'll investigate the version changes

[chcosta]

@riarenas , where are we w.r.t. flowing CLI version to repos via Arcade?

@alexperovich , do we have telemetry available which tells us the current version of the dotnet sdk Arcade repos are using?

[riarenas]

@riarenas , where are we w.r.t. flowing CLI version to repos via Arcade?

Waiting for final PR feedback on dotnet/arcade-services#294 and then will need a Maestro production rollout.

[jonfortescue]

@chcosta @riarenas @alexperovich -- what is the status on this? can it be merged?

[chcosta]

I need to further validate the MSBuild versioning changes based on recent conversations with Eric St. John. This issue is top of mind and I'll make progress soon.

[chcosta]

@ericstj , I'm internalizing your feedback as meaning that we have an upper limit on the version of NuGet we use which is defined by the MSBuild deps file provided with the sdk. So, we can upgrade the version, but realistically, we can never build against the "source build" version of NuGet and instead we should handle these as we handle other dependencies provided by the host (build against a reference package). Is that correct?

Are the MSBuild versions in a similar category; meaning we should build against a reference package rather than trying to consume the latest build version?

I looked at the issues you linked, and they provide additional context, but unfortunately the builds linked in the issues have expired so I'm not certain where the problems surfaced (ie, a good place to go to validate I'm not causing the same type of regressions that were previously fixed).

[ericstj]

All the failure modes are cases where tasks fail to load because they reference a version of a dll higher than the one pinned in the MSBuild.deps.json file. Depending on which MSBuild you're trying to run on you need to make sure your dependencies aren't higher than the versions pinned by that MSBuild. Building against a reference package may work, but it may not since many of these task DLLs redistribute their dependencies.

[chcosta]

What is your recommend for how to handle this? Is this something we can even enforce? This is sounding less like an Arcade issue, and more like something we need to drive from the source build perspective.

@dseefeld , how would you like to handle this category of issues?

[chcosta]

@dseefeld , can we get reference packages for Microsoft.Build.* and NuGet.* assemblies? These seem to fall into the category of dependencies which are provided by the host (.NET Core Sdk). Does that seem reasonable?

[dseefeld]

@chcosta I can look into adding them as reference packages.

[chcosta]

PlatformAbstractions and CodeAnalysis as well.

What's the best way to validate that, at that point, we will be compliant w.r.t. prebuilts?