Add compatibility attribute when linker is used as analyzer tool

[marek-safar]

This is a proposal to add global assembly attribute for assemblies which linker processes as an analyzer. The attribute could be something like ILLinkCompatibility ("{mode}", generated: true) which would indicate that during the assembly processing illinker didn't find any problems and are safe to be linked. The rules what it means exactly would need to be defined but it should include things like no warnings from flow-analysis, no usage of linker incompatible APIs, etc. The linker would not add this attribute if added manually by the user.

The mode value would reflect the highest linker compatibility level and would make to linker actions, specifically

• link
• copy
• copyused

The usage scenario would then be for any NuGet packages to be automatically linker enabled without doing any manual work on packages or references level.

[eerhardt]

I considered opening a new issue for the write-up below, but I felt this proposal is close enough to this discussion, that I decided to add it to this issue:

Problem

With Blazor WASM in .NET 5 we shipped an untrimmed System.IO.Pipelines.dll assembly in the default Blazor app. See dotnet/aspnetcore#27782. In 3.2, Blazor had the rule any assembly whose name starts with System. will get trimmed. Moving to the official .NET SDK in 5.0 changed this logic. By default, the .NET SDK will only trim assemblies that are shipped in the Microsoft.NETCore.App "runtimepack" (or think "Shared Framework"). There are a number of System. assemblies that aren't packaged into the Microsoft.NETCore.App "runtimepack", but instead dotnet/runtime only ships them as a NuGet packages. For example:

• System.CodeDom (in WindowsDesktop shared fx)
• System.Composition.*
• System.Configuration.ConfigurationManager (in WindowsDesktop shared fx)
• System.Diagnostics.EventLog (in ASP.NET shared fx)
• System.Diagnostics.PerformanceCounter
• System.DirectoryServices (in WindowsDesktop shared fx)
• System.Drawing.Common (in WindowsDesktop shared fx)
• System.Formats.Cbor
• System.IO.Packaging (in WindowsDesktop shared fx)
• System.IO.Pipelines (in ASP.NET shared fx)
• System.IO.Ports
• System.Management
• System.Net.Connections
• System.Net.Http.WinHttpHandler
• System.Net.WebSockets.WebSocketProtocol
• System.Reflection.MetadataLoadContext
• System.Resources.Extensions (in WindowsDesktop shared fx)
• System.Runtime.Caching
• System.Security.Cryptography.Pkcs (in ASP.NET & WindowsDesktop shared fx)
• System.Security.Cryptography.Xml (in ASP.NET & WindowsDesktop shared fx)
• System.Security.Permissions (in ASP.NET & WindowsDesktop shared fx)
• System.Threading.AccessControl (in WindowsDesktop shared fx)
• System.Windows.Extensions (in ASP.NET & WindowsDesktop shared fx)

If a trimmed app references any of the above assemblies that come from dotnet/runtime, they won't be trimmed by default.

Solution alternatives

We should consider a model where a NuGet package can some how opt into being trimmed by default. That way a user of a package doesn't have to "guess" if an assembly is safe for trimming, and having to mark the assembly as "trimmable" themselves.

The above is one proposal for how to do this. However, with the complete system we have today, we will have some problems with the assembly level attribute. The reason being that the .NET SDK marks every assembly that isn't marked as "trimmable" as -p copy on the command line. It wouldn't be a great design for the assembly attribute to override a command line arg. So we would either need to teach the SDK targets a different command line to invoke the ILLinker, or we would need to teach the SDK targets to crack open and inspect each assembly for the attribute.

Another potential issue with the top post's proposal is generating an assembly level attribute dynamically needs to be done in such a way that it works with "deterministic" builds. Since we don't have the ILLinker used as an analyzer tool yet, I'm not sure how this would work - if it would be a pre or post compile step. Just something to ensure we handle correctly.

Another alternative is to use the MSBuild extensibility we added in 5.0: dotnet/sdk#12035. In this solution, each NuGet package would need to ship a .targets file that hooks into the PrepareForILLink target, and marks themselves as themselves as TrimMode=link. While this can work, it feels unnecessarily complex and too much of a pain to ask NuGet library authors to ship a pre-canned .targets file.

Another alternative would be to add a new flag to the NuGet nuspec file that says "all the managed libraries in this package should be opted into trimming by default". We have similar metadata in NuGet packages - for example the serviceable attribute. However, it might not be great to add an ILLinker specific attribute directly into a nuspec file. We could also consider having a "sentinel" file in the nuget package that the SDK sees, and uses as "this assembly is trimmable".

Thoughts

I think the assembly level attribute approach has the best possibility of working. However, we will need to figure out how to make it work correctly with the SDK targets and command line options. This may drastically change the way we invoke the ILLinker today.

We can take baby-steps to get there:

1. Define the new assembly level attribute in dotnet/runtime
2. Modify the mono/linker to respect the new attribute in such a way that the SDK targets can invoke the ILLinker and attributed assemblies get trimmed by default.
3. Modify the dotnet/sdk to pass the newly formulated command line args to the ILLinker.

Then eventually, when we have the ILLinker running as an analyzer, we can figure out how to dynamically generate this attribute, if necessary.

I think we need to figure out a path forward here in .NET 6. There are more libraries that need to be trimmed by default than what is in the Microsoft.NETCore.App runtimepack today.

cc @vitek-karas @sbomer @MichalStrehovsky

[MichalStrehovsky]

Do we expect that going forward we'll keep the behavior that framework assemblies get trimmed by default, and everything else gets rooted?

Trimmability is not a property of an assembly - attribution that declares "I don't dynamically use any of my own code" is not very useful - there might still be code in another assembly that dynamically uses code in this assembly.

We applied such rule to framework assemblies because maybe it's a bit less likely that someone will be reflecting on the framework, but it's just a heuristic with no guarantees of actually helping. There are assemblies within framework where this is more likely to work, but also assemblies where it's less likely (e.g. assemblies that have classes that might be considered "data that is likely to be reflection-serialized" vs assemblies that have mostly code in them and don't represent data/state).

For .NET 6 I would hope we'll look more into directions where linker can prove correctness, as opposed to investing in heuristics that may result in broken apps.

[marek-safar]

The reason being that the .NET SDK marks every assembly that isn't marked as "trimmable" as -p copy on the command line. It wouldn't be a great design for the assembly attribute to override a command line arg.

I agree this setup is not ideal but we can certainly improve it. My suggestion would be to not treat the shared framework assemblies differently to any other library. If we standardized to assembly-level attribute we can add it to all libraries including shared framework assemblies. Then we don't pass any -p option to linker and leave the default behaviour to the attribute presence. The assembly link action would still be overrideable with existing -p if the user wants.

There is already attribute for that called LinkerSafeAttribute which we'll have to migrate to .NET6 for compatibility. I'm not saying we have to use this one but it'd make the migration for mobile easier.

Do we expect that going forward we'll keep the behavior that framework assemblies get trimmed by default, and everything else gets rooted?

No, we should make it possible to trim any library and in .NET6 we want to trim a couple of SDKs fully as well.

For .NET 6 I would hope we'll look more into directions where linker can prove correctness, as opposed to investing in heuristics that may result in broken apps.

I don't think it would be a good experience to run analyzer before linking all the times on libraries which are "static" for the end developer. We should run the verification at the end of the library build, that means once. If there are no issues detected it could be automatically marked as fully trimmable. Presumably, this could be done during build time via one analyzer but this probably needs more thoughts and it should not block us to allows this to be achieved manually.

[vitek-karas]

To me it seems we're mixing two concepts here:

• Asking SDK/linker to fully trim a subset of assemblies because we've found that this is likely to work and helps with size. The discussion is how to define the set (hardcoded in SDK, some external annotation, or internal (assembly attribute) annotation). Note that this is in all cases heuristic - just like @MichalStrehovsky points out above. This has basically nothing to do with correctness.
• Somehow annotate assembly that it doesn't produce warnings on its own - thus potentially saving work for the linker. If this would work properly it's a purely perf optimization. Again no impact on correctness really. We would obviously need numbers for this, but the analysis cost on its own is probably small - linker still has to do most of its work in constant propagation and marking on all assemblies, regardless of such annotation.

Blazor and Xamarin both want to use the ability to define a set of assemblies to trim - as a heuristic to improve application sizes without breaking them too much.

We were hoping that in 6 we would be able to invest into framework and some nugets to actually fix the correctness issue - which is not addressed by any of the above. There will be some work, but probably not enough to get us to a place where we can have true correctness for some end to end scenario (like Blazor or Xamarin). As such it's likely we will need to keep the heuristics around even for 6.

Based on the description above - I would stay away from names like "linker safe", "trim compatible" or anything similar for the proposed attribute. The attribute would be a technical way how to define the set of assemblies to apply aggressive trimming to in our heuristic scenarios. Nothing more, or less. I can see a world where we add it to even an assembly which still produces some linker warnings on its own, although it would be nice to not do that.

Let me reiterate: adding such attribute to an assembly doesn't make it safe to trim - the question if something is safe to trim or not can only be answered globally, not on per-assembly level.

So maybe a name like DefaultTrimAction("link") or similar.

[vitek-karas]

As for what the SDK should do - I think the current behavior is weird around assemblies like System.IO.Pipelines - at least in the Blazor case. It passes in -a System.IO.Pipelines which will root everything in that assembly AND annotate it with "copy" action. It then passes -p copy System.IO.Pipelines which will give it the "copy" action again.

The really "weird" part comes for the rest of the assemblies. The default action is set to "link" and then it singles out every assembly it doesn't know about (anything but frameworks). That is a really weird approach given that "link" is the most aggressive action. I would do it the other way round - default to the "safest" action, so probably "copy" and then single out framework assemblies with -p link System.Xml.dll.

Once we have the attribute discussed here, we could omit the -p and rely on the assembly attributes instead. The priority should definitely be: -p > attribute > -c/-u

If I understand how actions work in the linker, the -a should only be used on the application assembly itself. Since it's meant to mark "entry points".

[marek-safar]

So maybe a name like DefaultTrimAction("link") or similar.

I like the name or we could go with DefaultTrimMode to match msbuild property.

[sbomer]

I see the action as a decision made about how to link a particular assembly at link time - not a property of the assembly. After all, we want different actions for the wasm runtime than for the desktop runtime. As such I find it a bit strange to bake it in as an attribute.

I would do it the other way round - default to the "safest" action, so probably "copy" and then single out framework assemblies with -p link System.Xml.dll
Once we have the attribute discussed here, we could omit the -p and rely on the assembly attributes instead.

The issue with this is that the global TrimMode which can be set in the project file needs to affect the default for trimmed assemblies - it doesn't make sense for it to be a default for non-trimmed assemblies. If we do this with -p, it would override the action in the assembly attributes.

Putting the action in an assembly attribute would also require us to crack open every assembly just in case its action is copy, which goes against the direction we are heading with lazy loading. I would suggest not supporting copy in the assembly attribute.

-a should only be used on the application assembly itself. Since it's meant to mark "entry points".

We pass -a for non-trimmed assemblies to ensure that their dependencies are also kept.

[sbomer]

Putting the action in an assembly attribute would also require us to crack open every assembly just in case its action is copy

I realized it's worse than this - if we want to keep assemblies that haven't opted into trimming via an attribute, we will need to scan them all - regardless of how we pass options on the command-line.

Considering some scenarios where SDK options interact with the attributes:

• Developer sets <TrimMode>link</TrimMode> to run the linker in a more aggressive mode for trimmed assemblies

  To support this, there needs to be an attribute opt-in that respects the default TrimMode (-c/-u). Something like [DefaultTrimMode("default")] with a better name. :)

• Developer sets <IsTrimmable>true</IsTrimmable> metadata on an assembly to opt it into trimming.

  Shouldn't change the action if the assembly specifies [DefaultTrimMode("<action>")].
  Should use TrimMode if the assembly doesn't specify [DefaultTrimMode("<action>")].
  We would need a new command-line option for this. Maybe -p default <assembly>.

• Developer sets <TrimMode>link</TrimMode> metadata on an assembly to opt it into trimming.

  Passes -p link <assembly>. Overrides any action in the attribute.

• Assemblies without an attribute or IsTrimmable/TrimMode metadata are expected to get rooted and copied

  Instead of pre-scanning for attributes in the SDK and passing -p copy <assembly> and -a <assembly> for everything without an attribute, the linker should know about these defaults.

I'd like to keep this as simple as possible. I think that in the most common scenarios, a developer (framework author or external) simply wants to enable trimming of their assembly or of an assembly dependency of their app. Why don't we just support an unparameterized attribute like [IsTrimmable] (name TBD) that doesn't specify the action? If later we find a need to specify the action we can always extend it to take a string.

The semantics would be that explicit actions (-p) have the highest priority, and otherwise [IsTrimmable] assemblies get the TrimMode behavior. <IsTrimmable> metadata can continue passing -p actions (we don't need -p default).

Either way we still need the ability to root and keep everything that isn't opted into trimming.

[marek-safar]

To summarize your proposed logic the linker will be altered to follow these rules

1. if Assembly is listed with -p option on the command line
   -> is always taken for the assembly
2. then if assembly includes global IsTrimmable attribute
   -> assembly action is link
3. then if the global default action is set for the illink via command line (we merge -c and -u options into single new option)
   -> assembly uses the new global option value as the action
4. then fallback to a hardcoded value
   -> we set copy to the assembly action

Does it sound right?

[sbomer]

I would clarify a couple points. I think it's the same as what you are saying, but I want to be sure:

2. Assembly includes global IsTrimmable attribute
   -> assembly action is the default action if given on the command line, otherwise link
3. Global default action is set for the illink via command line (we merge -c and -u options into single new option)
   -> assembly uses the new global option value as the action if it has IsTrimmable, otherwise it getscopy behavior