Certain projects are failing to build with AOT and LLVM enabled against d15-6 on Windows

[pjcollins]

I'm seeing a new failure on Windows when compiling certain projects with both AOT and LLVM. The same error does not reproduce for me on macOS, and appears to be newly introduced in 15.6.

Steps to Reproduce

1. Download ApiDemo - https://github.com/xamarin/monodroid-samples/tree/master/ApiDemo
2. msbuild /p:Configuration=Release /t:SignAndroidPackage /p:AotAssemblies=true /p:EnableLlvm=true ApiDemo.csproj

Expected Behavior

The project builds successfully with AOT and LLVM enabled.

Actual Behavior

[aot-compiler stdout] Mono Ahead of Time compiler - compiling assembly C:\agent\_work\r2\a\monodroid-samples\ApiDemo\obj\Release\android\assets\shrunk\Mono.Android.Export.dll (TaskId:348)
  [aot-compiler stdout] AOTID 9B00228E-5ECD-7F32-1753-92A543C073ED (TaskId:348)
  [aot-compiler stdout] Executing opt: "C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\opt" -f -targetlibinfo -no-aa -basicaa -notti -instcombine -simplifycfg -inline-cost -inline -sroa -domtree -early-cse -lazy-value-info -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -loop-unswitch -instcombine -scalar-evolution -loop-simplify -lcssa -indvars -loop-idiom -loop-deletion -loop-unroll -memdep -gvn -memdep -memcpyopt -sccp -instcombine -lazy-value-info -correlated-propagation -domtree -memdep -adce -simplifycfg -instcombine -strip-dead-prototypes -domtree -verify -o "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.opt.bc" "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.bc" (TaskId:348)
  [aot-compiler stdout] Executing llc: "C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc"  -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_Mono_Android_Export_eh_frame -relocation-model=pic -o "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp-llvm.s" "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.opt.bc" (TaskId:348)
  [aot-compiler stderr] Stack dump: (TaskId:348)
  [aot-compiler stderr] 0.	Program arguments: C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_MonoDroid_ApiDemo_eh_frame -relocation-model=pic -o C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp-llvm.s C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp.opt.bc  (TaskId:348)
  [aot-compiler stderr] 1.	Running pass 'Function Pass Manager' on module 'C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp.opt.bc'. (TaskId:348)
  [aot-compiler stderr] 2.	Running pass 'Greedy Register Allocator' on function '@MonoDroid_ApiDemo_ApiDemo_GetDemoActivities_string' (TaskId:348)
  [aot-compiler stderr] AOT of image C:\agent\_work\r2\a\monodroid-samples\ApiDemo\obj\Release\android\assets\MonoDroid.ApiDemo.dll failed. (TaskId:348)

Version Information

https://gist.github.com/pjcollins/842d3a55d49841a4fa955cd403af1549
Android NDK 15.2.4203891

Log File

http://xqa.blob.core.windows.net/gist/report-7d4d56fee1504d14818b0cef6538694e.txt

[kumpera]

Log output for the AOT task looks incomplete.

Could you manually run the AOT commands as stated in the log files and provide the output?
You might have to change project settings to keep temp files.

[vargaz]

Could you upload the obj/Release directory somewhere ?

[JKennedy24]

also seeing this on my automated build machine after updating to VS 15.6

[pjcollins]

Another item to note is that this does not occur when the project only supports armeabi-v7a, from what I have seen so for I need to also enable arm64-v8a to reproduce.

@vargaz I've uploaded the intermediate output here.

@kumpera Is there a way for me to get more output aside from msbuild /v:diag? If not, should I file a new issue for enhancing error output for the AOT task for cases such as this? I am assuming that we are not getting any additional useful output because llc.exe is crashing after invoking the first failing command such as:

 C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_System_eh_frame -relocation-model=pic -o C:\Users\XAMARI~1\git\MONODR~1\ApiDemo\obj\Release\aot\ARM64-~1\System.dll\temp-llvm.s C:\Users\XAMARI~1\git\MONODR~1\ApiDemo\obj\Release\aot\ARM64-~1\System.dll\temp.opt.bc 

Results in llc.exe aborting. VS also hangs and then reboots if I reproduce this from within the IDE:

The thread 0x2240 has exited with code 0 (0x0).
Unhandled exception at 0x0091DF49 in llc.exe: 0xC0000005: Access violation reading location 0x00000010.
Exception thrown at 0x77935BBE (ntdll.dll) in llc.exe: 0xC0000005: Access violation writing location 0xFFFFFFFC.

[vargaz]

I can't reproduce on osx with the files from the Release directory, so it might be a problem with the llc executable itself, not with the input to it.

[vargaz]

The same file causes the llc.exe from 15-6 to fail, while the llc.exe from 15-5 works fine. The llvm version in 15-5/15-6 appears to be the same, so it might be a change in the way llc.exe is compiled.

[jonpryor]

@vargaz: The MXE version was bumped between d15-5 and d15-6.

Perhaps this is an MXE and/or compiler bug?

[jonpryor]

@dellis1972: Don't we have any tests which try building with AOT+LLVM enabled?

If we don't, please add one. :-)

[vargaz]

Thats most likely the culprit.

[jaykrell]

Is this "c000005" rendition of the problem easy to setup and run on Windows?
Under a debugger this will stop right away, and we have reverse debugging..
This is equivalent to sigsegv.