File path issue with ProjectReference + SourceLink

[TimHess]

Hello,

We're using coverlet in our project and we've been experiencing one of the known issues with the msbuild integration, so I'm working on switching to collectors in this PR. Almost everything is working correctly now, except that when tests run with a .NET Framework target and project references are involved, the path for files in the referenced project shows up as the local path instead of the sourcelink path like this.

Steeltoe is a pretty large set of packages and interconnected pieces, I can provide some more specific details if that will help but first I wanted to check if this is more likely to be an issue here or with SourceLink

[TimHess]

I was able to get past this issue by setting <DebugType>full</DebugType>, which resulted in all file paths in the report showing a local path on the build agent's file system.

I don't know that I care if the code coverage report shows a local path on the build server or the SourceLink path, so long as files don't show up twice. I am interested in hearing if there's a way that it should work or if there's something I could/should be doing differently in this space, thanks

[MarcoRossignoli]

Thank's for reporting this.

Do you get this issue only on .NET Framework?No issues on .NET Core?

[TimHess]

That's correct - on that branch of that repo, most tests run for net461, netcoreapp2.1 and netcoreapp3.1. I isolated the issue by just running the tests in Common.sln to get to that conclusion.

I made a repro that was failing the same way but I must have made some inadvertent change and now it all seems to be working and showing the sourcelink paths no matter what I do

This is how I've been running the tests and generating a report from the results:

dotnet test --blame -c Release --settings coverlet.runsettings --logger trx --results-directory C:\temp\testresults /p:TF_BUILD=true
reportgenerator -reports:c:\temp\testresults\**\coverage.opencover.xml -targetdir:c:\temp\testresults\CodeCoverage -reporttypes:"HtmlInline_AzurePipelines;Cobertura" -plugins: -assemblyfilters:+* -classfilters:+* -filefilters:+* -verbosity:Info

[TimHess]

It turns out my workaround isn't a real solution as now nuget.org won't validate the symbols package, so I'm back to trying to figure this out again.

Here are the steps to see the issue in the Steeltoe codebase:

git clone https://github.com/SteeltoeOSS/Steeltoe.git
cd Steeltoe
git checkout coverage
dotnet build .\src\Common\Common.sln -c Release /p:TF_BUILD=true
dotnet test .\src\Common\Common.sln --no-build -c Release --settings coverlet.runsettings --logger trx --results-directory C:\temp\testresults /p:TF_BUILD=true

# and then optionally
reportgenerator -reports:c:\temp\testresults\**\coverage.opencover.xml -targetdir:c:\temp\testresults\CodeCoverage -reporttypes:"HtmlInline_AzurePipelines;Cobertura" -plugins: -assemblyfilters:+* -classfilters:+* -filefilters:+* -verbosity:Info

I was pretty sure the issue only happened with net461, but today it looks like it's happening for netcoreapp3.1 also. netcoreapp2.1 doesn't seem to have the issue, but I'm also missing results for the base library so it might not be that it is correct but rather something else is wrong. Nothing jumps out at me as a relevant difference between my simple sample and Steeltoe, but I must be missing something. Also it looks like I forgot to mention that deterministic is enabled (in both the simple sample and Steeltoe)

[TimHess]

Here are some logs in case they can help
logs.zip

[clairernovotny]

Couple thoughts:

1. Can you use either embedded or portable PDB types? I prefer embedded as no separate snupkg is needed. NuGet.org doesn't support Full symbols and full symbols only work on Windows.
2. Use the coverlet.collectors package reference and the 16.7+ MSTest SDK. You'll also need this in your targets until Coverlet updates to use the built-in target: https://github.com/dotnet/reactive/blob/main/Rx.NET/Source/Directory.build.targets#L45-L53
3. You can see how Rx runs the tests. Note that Rx multi-targets and has tests on .NET Core and .NET Framework, so it should be similar to what you're doing. It uses embedded symbols: https://github.com/dotnet/reactive/blob/main/azure-pipelines.rx.yml#L77
4. The full pipeline is in that file if it's helpful to you. It generates multiple code coverage files per project (due to the multi targeting) that is merged into a single one published to DevOps.

[TimHess]

@clairernovotny thanks for the response

1. I don't have a preference, we weren't previously specifying what type was produced at all, it just needs to work. Unfortuntely, switching to portable or embedded doesn't seem to have had any effect on the problem that brought me here
2. Already did all of that
3. I was wrong when I said it was related to the target framework
4. Thanks, we're already doing that too.

To restate the problem, the code coverage reports for tests on projects that include a projectreference are reporting a local path for the ProjectReference'd files , where the library being directly tested will use sourcelink paths... Please review the attached file and note the difference between the paths for files under Steeltoe.Common vs Steeltoe.Common.Net

coverage.opencover.txt

[clairernovotny]

Coverlet should be getting the map for each project reference here to generate a multi map of project reference to local location:

coverlet/src/coverlet.collector/build/netstandard1.0/coverlet.collector.targets

Lines 25 to 32 in 8cf2b3c

	<Target Name="ReferencedPathMaps" BeforeTargets="CoreCompile" DependsOnTargets="ResolveProjectReferences" >
	<MSBuild Projects="@(AnnotatedProjects->'%(FullPath)')"
	Targets="CoverletGetPathMap"
	Properties="TargetFramework=%(AnnotatedProjects.NearestTargetFramework)"
	SkipNonexistentTargets="true">
	<Output TaskParameter="TargetOutputs"
	ItemName="_LocalTopLevelSourceRoot" />
	</MSBuild>

Is there a CoverletSourceRootsMapping file in your output directory that you can attach here?

[TimHess]

I don't see one

[TimHess]

I was looking in the wrong output and search failed me... found one
CoverletSourceRootsMapping.zip

[TimHess]

OK, so I found something else weird.. This looks like it's happening in test runs for Steeltoe.Common.Net, but not Steeltoe.Common.Http. These both target netstandard2.0, have tests running the same target frameworks and both depend on the same base library... I have no idea what could be different between the two that would cause this behavior to be different

Here are my latest CoverletSourceRootsMapping files and opencover reports
latestresults.zip