ci: use publish code coverage results v1

[QxBytes]

Reason for Change:

Use publish code coverage results v1 until v2 bug is fixed. Code coverage v1 does not support multi file coverage reports and merging the two files does not give the expected result (depending on the GOOS during gocov, os-specific files are included/excluded from the report).

Will re add the windows ut coverage after v2 fix is out.

Green run: https://msazure.visualstudio.com/One/_build/results?buildId=123636014&view=results

Issue Fixed:

Requirements:

• uses conventional commit messages
• includes documentation
• adds unit tests
• relevant PR labels added

Notes:

[QxBytes]

/azp run Azure Container Networking

[azure-pipelines]

No pipelines are associated with this pull request.

[Copilot]

Pull Request Overview

This PR updates the CI pipeline for Azure Container Networking to use Publish Code Coverage Results v1 instead of v2 due to multi-file report limitations and removes Windows coverage steps from the pipeline.

• Removed Windows-specific pipeline tasks (artifact download, coverage conversion, and report migration).
• Downgraded the publish code coverage task to version 1 and updated its configuration accordingly.

Comments suppressed due to low confidence (1)

.pipelines/templates/run-unit-tests.yaml:75

• Removal of Windows coverage tasks means only Linux builds are covered; ensure this change aligns with the intended CI coverage requirements.

-              GOOS=windows gocov convert windows-coverage.out > windows-coverage.json

[QxBytes]

/azp run Azure Container Networking PR

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[pjohnst5]

I think we should keep the windows test results, and combine them into one file (or does that not work?)

[pjohnst5]

I think we should maybe consider combining these coverage files, like so:

mv linux-coverage.out total-coverage.out # rename to just total-coverage.out
sed -i "1d" windows-coverage.out  # removes first "mode: set" line from windows-coverage.out file
cat windows-coverage.out >> total-coverage.out
gocov convert total-coverage.out > total-coverage.json
gocov-xml < total-coverage.json > total-coverage.xml

This will combine them into one
If the windows coverage was there before, I don't think we should get rid of it

[QxBytes]

yes I tried to use that method before using the multi file method but it seems like the coverage xml changes based on whether I GOOS=windows or GOOS=linux with gocov:
GOOS=windows gocov convert total_ut_coverage.out > total_ut_coverage.json will not include files like os_linux in the json/xml
GOOS=linux gocov convert total_ut_coverage.out > total_ut_coverage.json will not include files like os_windows in the json/xml
The coverage % also differ depending on whether it is GOOS=windows (.4743) or GOOS=linux (.4718)

[timraymond]

We really need to pin down whether merging coverage like this is a "set union" operation or not. It doesn't seem like it is from what @QxBytes reports... that may be the source of some confusing results.

[pjohnst5]

Alright yeah @QxBytes I see different behavior when:

• Specifying GOOS=windows when converting the windows_coverage.out as opposed to GOOS=linux
• Concatenating the windows_coverage.out to linux_coverage.out doesn't really add up
  • It only adds like ~2k lines to lines_covered, even though windows_coverage.xml showed ~5k lines covered (maybe they have overlap or something, like a common library between the two of them that gets tested twice, we already know if we have two duplicate lines in code coverage, the higest value is taken into account, regardless or ordering)

I'm attaching my *-coverage.xml and resulting coverage reports folders

Alexander, would you be able to see why the numbers are different? I think we don't understand fully yet why, and we may be able to get a more comprehensive look at the coverage overall if we can understand how combining them works
coverage_tests.zip

linuxcoverage:

wincoverage (GOOS was linux; default):

totalcoverage (concatenating windows_coverage.out to linux_coverage.out and running conversion commands with GOOS=linux (default):

winoscoverage (GOOS=windows on windows_coverage.out):

totaloscoverage(GOOS=windows after concatenating windows_coverage.out to linux_coverage.out):

These are the covered lines of each run:

Linux covered:12101
Win covered: 5044
Total covered: 12279

Win os covered: 5939
Total os covered: 10989

[QxBytes]

They are inherently different based on whether you run the go cov with GOOS=windows or GOOS=linux. I believe based on the GOOS gocov will only take the lines that are for that os or are common to both-- so total w/ windows goos will take the .out file and only get the coverage that would be visible if it were a windows os, and same with linux. There isn't a set union based on what I can see. This is a limitation of gocov itself and we may be able to get multi-os reports again when the Publish Code Coverage v2 issue is fixed.

[pjohnst5]

Okay yeah that's fine, can you just leave a #TODO in there somewhere mentioning to add back in Windows code coverage when PCCRv2 works with BQC?

[QxBytes]

/azp run Azure Container Networking PR

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).