Reflection-free mode support

[kant2002]

Currently Microsoft.Data.SqlClient cannot work when compiled in NativeAOT and Reflection-Free mode.

dotnet/aspnetcore#31561 (comment)

Would be good that this library would be more friendly for this kind of scenarios.

[Wraith2]

Can you give me a sqlclient only repro including the incantations needed to get it compiled to aot and failing? with that I can investigate what reflection is being used for and either make it static or identify the required linker hints.

[JRahnama]

@kant2002 thanks for reaching out to us here. Enhancements are based on popularity of the matter. We have more urgent/requested issues or enhancements to solve at the moment. I cannot provide a timeframe on this topic until it becomes a popular/requested item in the community. We will look into this in future.

@Wraith2 I really appreciated your offer. If you can take over this based on a provided application we can go over the PR and merge it if it has no conflicts with other in progress PRs or future plans.

Thanks.

[cheenamalhotra]

Hi @kant2002

A repro would definitely be helpful to understand gaps for Native AOT compilation support but I'm thinking one of the reasons it's failing is due to our Native C++ DLL dependency on Windows (SNI DLLs). As mentioned in documentation for ReadyToRun compilation, C++/CLI binaries are not eligible.

[LLT21]

As the author of the original issue in dotnet/aspnetcore#31561 (comment) I have created a repository https://github.com/LLT21/NativeSQL.

Please add in the ConnectionStrings.cs class 2x the database with user and password you can connect to (for the odbc, the driver has to be installed): https://github.com/LLT21/NativeSQL/blob/1d074421a6772176adce97140745e2f2b2e4c11e/ConnectionStrings.cs#L5.

In the main program.cs change 2x the table name in: https://github.com/LLT21/NativeSQL/blob/1d074421a6772176adce97140745e2f2b2e4c11e/Program.cs#L23 and
https://github.com/LLT21/NativeSQL/blob/1d074421a6772176adce97140745e2f2b2e4c11e/Program.cs#L48
to a table in your database;

Also in the main program.cs change 2x the column name in:
https://github.com/LLT21/NativeSQL/blob/1d074421a6772176adce97140745e2f2b2e4c11e/Program.cs#L32 and
https://github.com/LLT21/NativeSQL/blob/1d074421a6772176adce97140745e2f2b2e4c11e/Program.cs#L57
to a column in your table.

Luc

[LLT21]

The error I receive at native execution is:
Unhandled Exception: EETypeRva:0x011CA2A0: TypeInitialization_Type_NoTypeAvailable
---> EETypeRva:0x011C9578: Reflection_Disabled

Michal Strehovsky has helped a lot in solving these kind of issues in the aspnetcore https call

[Wraith2]

The repro works (in that it fails with that same message) for me. I can also confirm that the Microsoft.Data.SqlClient.SNI.dll is copied to the output directly @cheenamalhotra and I was hoping that would be the problem.

Interestingly when i tried it with debug mode the ILCompiler crashed using

E:\Programming\csharp7\dev\NativeSQL-main>dotnet publish -r win-x64 -c Debug
Microsoft (R) Build Engine version 16.10.0-preview-21175-01+afd0b6210 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored E:\Programming\csharp7\dev\NativeSQL-main\nativesql.csproj (in 40.25 sec).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  nativesql -> E:\Programming\csharp7\dev\NativeSQL-main\bin\Debug\net5.0\win-x64\nativesql.dll
  Generating native code
  Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
     at ILCompiler.RootingHelpers.TryGetDependenciesForReflectedType(DependencyList& dependencies, NodeFactory factory, TypeDesc type, String reason)
     at ILCompiler.RootingHelpers.TryGetDependenciesForReflectedField(DependencyList& dependencies, NodeFactory factory, FieldDesc field, String reason)
     at ILCompiler.DependencyAnalysis.DynamicDependencyAttributeAlgorithm.AddDependenciesDueToDynamicDependencyAttribute(DependencyList& dependencies, NodeFactory factory, EcmaMethod method)
     at ILCompiler.UsageBasedMetadataManager.GetDependenciesDueToMethodCodePresenceInternal(DependencyList& dependencies, NodeFactory factory, MethodDesc method, MethodIL methodIL)
     at ILCompiler.DependencyAnalysis.CodeBasedDependencyAlgorithm.AddDependenciesDueToMethodCodePresence(DependencyList& dependencies, NodeFactory factory, MethodDesc method, MethodIL methodIL)
     at Internal.JitInterface.CorInfoImpl.PublishCode()
     at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL)
     at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodCodeNode methodCodeNodeNeedingCode, MethodIL methodIL)
     at ILCompiler.RyuJitCompilation.CompileSingleMethod(CorInfoImpl corInfo, MethodCodeNode methodCodeNodeNeedingCode)
     at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
     at System.Threading.ThreadPoolWorkQueue.Dispatch()
     at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
E:\Programming\csharp7\dev\.nupkg\microsoft.dotnet.ilcompiler\6.0.0-preview.5.21226.1\build\Microsoft.NETCore.Native.targets(282,5): error MSB3073: The command ""E:\Programming\csharp7\dev\.nupkg\runtime.win-x64.microsoft.dotnet.ilcompiler\6.0.0-preview.5.21226.1\tools\ilc" @"obj\Debug\net5.0\win-x64\native\nativesql.ilc.rsp"" exited with code -1073741819. [E:\Programming\csharp7\dev\NativeSQL-main\nativesql.csproj]

And I don't know if I should bother opening an issue for that in runtime since it's with the latest net6 preview tooling @MichalStrehovsky ?

[Wraith2]

The first problem is strings from embedded resources. We're using standard framework provided ResourceManager so I think that the advice from the aspnet thread about adding in a property called UsingResourceKeys and plumbing that in so that it interacts with the linker will work. If we do that azure connection strings aren't going to enable azure network resiliency because the endpoint keys aren't the uri's. So we need to get the actual resources or change how we're storing them, in this case they could probably be taken out of the string resources and made into embedded strings but is there a way to enable their use from the resources file?

Another problem is going to be that I think it's going to be trying to use GlobalizationInvariant mode and this library does not and cannot support that mode. We need to be able to load culture specific comparers to correctly decode strings coming from the server using their marked collation.

[MichalStrehovsky]

And I don't know if I should bother opening an issue for that in runtime since it's with the latest net6 preview tooling @MichalStrehovsky ?

That's probably dotnet/runtimelab#1027. It should be fixed now.

Note that this is for the very experimental reflection free mode which is more of a tech demo than anything else. Reflection is not a problem for AOT (not more than for trimming in general), except for these obscure modes.

[Wraith2]

Ok, if it's still WIP I'll leave alone. I've resolved the embedded resource issue and it's probably worth merging that code into SqlClient anyway so that it'll can work in future, it's not harmful to be included.

I've also found that we're going resource loading wrong through the entire library. We use the compiler generated strongly typed accessor to lookup the string and then pass that string to GetFormattedString which attempts to lookup the string as if it were a key when in fact it's already the value. I can address that in a PR as well.

I am still interested in this mode though because while it probably won't work because we use GetType() it currently won't open a connection and it doesn't crash, it just hangs with a native thread in FinalizerHelpers.cpp StartFinalize (i think) and I don't think that's right.

[kant2002]

If you consider not only no-reflection mode, current incarnation of SqlClient adds ~33Mb to the produced executable. This is maybe not an issue per-se, but can hint you in parts which are not very optimal and have too much dependencies on each other

[Wraith2]

I noticed that the produced binary was very large. But then it's a runtime-less single executable so it contains dependencies from the runtime as well. There are things like the globalization data which is 10Mib of icu data that can't be removed.

I haven't worked with AOT or trimming so it will take a while for me to get the hang of the process and tooling but I'll keep looking at it. Just don't expect quick results.

[kant2002]

I'm willing to spend some time into that too. If this project would be accepting discussion on that matter, I more then happy to help.