git2-ef5a385

[dogguts]

docfx crash report: System.AggregateException

docfx: 3.0.0-beta-01715-e1f0d1630+e1f0d163029f0c5892899380684e273d2ca7fcf8
dotnet: 3.1.201
x64: True
git: git version 2.20.1

repro steps

Run /home/dogguts/.dotnet/tools/.store/docfx/3.0.0-beta-01715-e1f0d1630/docfx/3.0.0-beta-01715-e1f0d1630/tools/netcoreapp3.1/any/docfx.dll build in /home/dogguts/aeon_gh/docs

callstack

System.AggregateException: One or more errors occurred. (The type initializer for 'Microsoft.Docs.Build.LibGit2' threw an exception.)
 ---> System.TypeInitializationException: The type initializer for 'Microsoft.Docs.Build.LibGit2' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'git2-ef5a385' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libgit2-ef5a385: cannot open shared object file: No such file or directory
   at Microsoft.Docs.Build.LibGit2.git_libgit2_init()
   at Microsoft.Docs.Build.LibGit2..cctor() in D:\a\1\s\src\docfx\lib\git\LibGit2.cs:line 19
   --- End of inner exception stack trace ---
   at Microsoft.Docs.Build.LibGit2.git_repository_open(IntPtr& repo, String path)
   at Microsoft.Docs.Build.GitUtility.IsGitRepository(String path) in D:\a\1\s\src\docfx\lib\git\GitUtility.cs:line 42
   at Microsoft.Docs.Build.GitUtility.FindRepository(String path) in D:\a\1\s\src\docfx\lib\git\GitUtility.cs:line 23
   at Microsoft.Docs.Build.Repository.Create(String path, String branch, String repoUrl) in D:\a\1\s\src\docfx\lib\git\Repository.cs:line 46
   at Microsoft.Docs.Build.Repository.Create(String path) in D:\a\1\s\src\docfx\lib\git\Repository.cs:line 32
   at Microsoft.Docs.Build.ConfigLoader.Load(String docsetPath, String outputPath, CommandLineOptions options) in D:\a\1\s\src\docfx\config\ConfigLoader.cs:line 51
   at Microsoft.Docs.Build.Restore.RestoreDocset(String docsetPath, String outputPath, CommandLineOptions options) in D:\a\1\s\src\docfx\restore\Restore.cs:line 47
   at Microsoft.Docs.Build.Build.<>c__DisplayClass0_0.<Run>b__0(ValueTuple`2 docset) in D:\a\1\s\src\docfx\build\Build.cs:line 26
   at System.Threading.Tasks.Parallel.<>c__DisplayClass32_0`2.<ForEachWorker>b__0(Int32 i)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica`1.ExecuteAction(Boolean& yieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
   at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException)
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](TSource[] array, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
   at Microsoft.Docs.Build.Build.Run(String workingDirectory, CommandLineOptions options) in D:\a\1\s\src\docfx\build\Build.cs:line 37
   at Microsoft.Docs.Build.Docfx.Run(String[] args) in D:\a\1\s\src\docfx\cli\Docfx.cs:line 76
   at Microsoft.Docs.Build.Docfx.Main(String[] args) in D:\a\1\s\src\docfx\cli\Docfx.cs:line 25

[dogguts]

dotnet global tool installed with;

dotnet tool install -g docfx --version "3.0.0-*" --add-source https://www.myget.org/F/docfx-v3/api/v2

Seems like https://github.com/dotnet/docfx/blob/v3-release/src/docfx/lib/git/LibGit2.cs is referencing a hard coded (custom-git-build) version of libgit2.
As workaround I made a symlink (in /usr/lib/x86_64-linux-gnu), 'resolving' the above crash:
$sudo ln libgit2.so.27 libgit2-ef5a385.so -s

Additional system information:

$lsb_release  -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 10 (buster)
Release:	10
Codename:	buster

$ uname -a
Linux dogguts 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.201
 Commit:    b1768b4ae7

Runtime Environment:
 OS Name:     debian
 OS Version:  10
 OS Platform: Linux
 RID:         debian.10-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.201/

Host (useful for support):
  Version: 3.1.3
  Commit:  4a9f85e9f8

.NET Core SDKs installed:
  2.1.805 [/usr/share/dotnet/sdk]
  3.0.103 [/usr/share/dotnet/sdk]
  3.1.201 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.17 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.17 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.17 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

[dogguts]

looking a bit further into this issue.
My RID is "debian.10-x64", and no native libraries are found (by docfx) for that runtime. A generic ("linux-x64") fallback is present, but isn't called/considered either.

docfx (v3) has a package reference to "LibGit2Sharp.NativeBinaries" (2.0.298"), but "LibGit2Sharp" seems to have a more advanced native library resolver; libgit2/libgit2sharp/.../NativeMethods.cs.
Perhaps an option to expand LibGit2.cs with some LibGit2Sharp functionality to support more (linux-)platforms out-of-the-box throught the generic fallback runtime?
Or use LibGit2Sharp altogether? Or use a newer version of the "LibGit2Sharp.NativeBinaries", wich seems to contain only 'generic' RIDs.

So, since docfx comes with its own native libgit2 libraries (through the "LibGit2Sharp.NativeBinaries" package); I removed the symlink (I made earlier so the system version of libgit2 could be used) and modified "docfx.deps.json" instead, added debian 10 RID to point to the debian-9 library;

          "runtimes/debian.9-x64/native/libgit2-ef5a385.so": {
            "rid": "debian.10-x64",
            "assetType": "native",
            "fileVersion": "0.0.0.0"
          },

btw. trying to use the "runtimes/linux-x64/native/libgit2-ef5a385.so" didn't work (didn't look further into it).

[yufeih]

According to libgit2/libgit2sharp#1703, LibGit2Sharp seems to have problem with debian 10 as well. The native library resolver comes with libgit2sharp may not resolve this problem.

[dogguts]

I found the issue and simple/easy solution.
The problem was not that debian 10 isn't explicitly supported. But the RID 'fallback' ".../linux-x64/native/libgit2-ef5a385.so" was missing a (native) dependency.

Solution (~TL;DR)

up the "LibGit2Sharp.NativeBinaries" version to at least 2.0.305 (i used 2.0.312)
libgit2/libgit2sharp.nativebinaries#96

LibGit2Sharp.NativeBinaries" (2.0.298)

"libgit2-ef5a385.so" still depends on libssl1.0.0 (and libcrypto1.0.0).
But, most (if not all) modern linux distributions don't, uhm, distribute libssl1.0.0 any more (~considered insecure).

~/LocalProjects/docfx/drop/.store/docfx/3.0.0/docfx/3.0.0/tools/netcoreapp3.1/any/runtimes/linux-x64/native$ ldd libgit2-ef5a385.so 
	linux-vdso.so.1 (0x00007fff0dad2000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc75b109000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc75b0e8000)
	libssl.so.1.0.0 => not found
	libcrypto.so.1.0.0 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc75af27000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fc75b485000)

LibGit2Sharp.NativeBinaries" (>= 2.0.305)

Removed the dependency to libssl (and libcrypto); libgit2/libgit2sharp.nativebinaries#96

~/LocalProjects/docfx/drop/.store/docfx/3.0.0/docfx/3.0.0/tools/netcoreapp3.1/any/runtimes/linux-x64/native$ ldd libgit2-6777db8.so 
	linux-vdso.so.1 (0x00007fff0c3db000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f4c8c9bb000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4c8c99a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4c8c7d9000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4c8cd2e000)

As an added bonus the RID's (and consequently the provided runtimes) for linux where cleaned up as well;
libgit2/libgit2sharp.nativebinaries@6844e70

[yufeih]

LibGit2Sharp is upgraded to the latest version.