Enable source-build for the ASP.NET Core shared framework

[natemcmaster]

See also dotnet/source-build#375.

The Microsoft.AspNetCore.App shared framework and many of dependencies are not currently supported in source-build. To enable source-build on this shared runtime, we expect the following changes will need to be made:

• Add a build flag to enable building only source-buildable assets. Not everything in the repo will support source-build
• Enabling compiling without any remote assets.
• Change our build scripts skip steps which require unavailable tools, such as NuGetPackageVerifier, xunit analyzers, etc.

Requirements

The following must be available via source-build:

• everything in the Microsoft.AspNetCore.App shared framework for Linux (currently excluding Windows, which requires Visual Studio to build)
• all assemblies in the Microsoft.AspNetCore.App targeting pack

What's not in source-build yet:

• test projects
• internal tools
• KoreBuild
• anything not in Microsoft.AspNetCore.App

[natemcmaster]

I've looked closer at implementing this. Here are some thoughts

Notes:

• Repos to add to source-build, in order
  1. aspnet/Extensions
  2. aspnet/AspNetCore-Tooling
  3. aspnet/AspNetCore
• It's unclear how our contributors and devs will know if a particular change would break source-build. I cannot identify a simple way to test source-build support as a pull-request check.
• Dependency flow for Maestro and dependency flow for source-build are slightly different (see https://github.com/dotnet/source-build/tree/master/Documentation/auto-dependency-flow). I think it should be possible to make both work without having duplicate dependency-flow systems, but concerned it may be fragile unless we have a good way to test this continually.
• We will need a fix or workaround for Building a targeting and runtime pack locally sdk#2910 before we can add aspnet/AspNetCore. Our build currently the build assumes the SDK is in $repoRoot/.dotnet/, but according to the Repo API, we may need to allow for external location and version of dotnet
• Convert to Arcade SDK #7280 - convert to Arcade. Done on 2/3 repos. This should be done first so we don't also need to convert KoreBuild to source-build
• The Repo API calls for supporting .rsp files as input to build.sh/cmd scripts (see https://github.com/dotnet/source-build/blob/master/Documentation/RepoApi.md#rsp-files). This may be challenging to implement in PowerShell/bash. We should look to see if we can copy what other repos have done.
• Source build for Windows will be very challenging due to our dependency on Visual Studio C++ tooling to build the IIS inprocess hosting .dll.
• As mentioned in OP, we do not intend to build every line of code in these repos. This means we need exclude the following when -DotNetBuildFromSource true is set
  • Visual Studio tooling projects (.vsix)
  • All NuGet packages, except those required to move bits between repos (in 3.0, targeting packs and runtime packs are bundled in the SDK and not necessarily consumed via NuGet)
  • All test projects and samples
  • All NodeJS and Java code

cc @Eilon @dseefeld