The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1113967	dotnet/runtime	Send tests to Helix (Unix)	Log
1113884	dotnet/runtime	Send to Helix	Log	dotnet/runtime#111178
1114128	dotnet/sdk	Run TestBuild Tests	Log	dotnet/sdk#49724
1113958	dotnet/sdk	Run Tests in Helix	Log	dotnet/sdk#49852
2765049	dotnet-sdk	Run Tests in Helix	Log	#52071
1113922	dotnet/sdk	Run TestBuild Tests	Log	dotnet/sdk#49724
1113921	dotnet/sdk	Run AoT Tests in Helix	Log	dotnet/sdk#50041
1113918	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#49822
2765019	dotnet-runtime	Run performance job script	Log
1113528	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79578
2764962	dotnet-dotnet-monitor	Build	Log
2764943	dotnet-sdk	Run Tests in Helix	Log	#52071
1113804	dotnet/sdk	Run AoT Tests in Helix	Log	dotnet/sdk#49828
1113780	dotnet/sdk	Run FullFramework Tests	Log	dotnet/sdk#49822
1113777	dotnet/sdk	Run FullFramework Tests	Log	dotnet/sdk#49724
1113791	dotnet/sdk	Run AoT Tests in Helix	Log	dotnet/sdk#50044
1113783	dotnet/sdk	Run AoT Tests in Helix	Log	dotnet/sdk#50041
1113768	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#49843
1113771	dotnet/sdk	Run AoT Tests in Helix	Log	dotnet/sdk#49852
1113770	dotnet/sdk	Run Tests in Helix	Log	dotnet/sdk#49853
1113314	dotnet/runtime	Send tests to Helix (Unix)	Log
1113712	dotnet/roslyn	Run Integration Tests	Log
1113733	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#49889
1113705	dotnet/roslyn	Run Integration Tests	Log
2764895	dotnet-roslyn	Test	Log
1113686	dotnet/roslyn	Run Integration Tests	Log
2764883	dotnet-roslyn	Test	Log
2764879	dotnet-roslyn	Test	Log
2764854	dotnet-sdk	Run AoT Tests	Log	#52073
2764851	dotnet-sdk	Run AoT Tests in Helix	Log	#52072
2764845	dotnet-sdk	Run AoT Tests in Helix	Log	#52070
2764846	dotnet-sdk	Run Tests in Helix	Log	#52071
2764719	dotnet-sdk	Run FullFramework Tests	Log	#51985
2764772	dotnet-roslyn	Test	Log
2764729	dotnet-dotnet-monitor	Build	Log
2764726	dotnet-efcore	Send job to helix	Log
2764717	dotnet-sdk	Run AoT Tests	Log	#52073
1113493	dotnet/roslyn	Run Integration Tests	Log
1113487	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79764
2764693	dotnet-sdk	Run Tests in Helix	Log	#52072
2764610	dotnet-dotnet	Build (centOSStreamContainer)	Log
1113358	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118306
1113304	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118360
1113389	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63099
1113322	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118361
1113382	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63111
1113316	dotnet/runtime	Send to Helix	Log
1113280	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118357
2764421	dotnet-dotnet	Run Binary Analysis Scan Optimization (1ES PT)	Log
2764394	dotnet-dotnet	Build (centOSStreamContainer)	Log
1112960	dotnet/roslyn	Run Integration Tests	Log
1112650	dotnet/runtime	Send to Helix	Log
2764121	dotnet-runtime	Send job to Helix (Unix)	Log
2764093	dotnet-dotnet	Build (azurelinuxX64MuslCrossContainer)	Log
2763939	dotnet-runtime	Run performance job script	Log
2763909	dotnet-orleans	Test	Log
2763908	dotnet-performance	Send job to Helix (Windows)	Log
1112343	dotnet/runtime	AOT compile CoreCLR tests	Log
1112288	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#117845
1112277	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#117997
1112282	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118241
1112191	dotnet/runtime	Send tests to Helix (Unix)	Log
2763711	dotnet-dotnet	Build (azurelinuxArm64CrossContainer)	Log
2763703	dotnet-performance	Send job to Helix (Windows)	Log
2763566	dotnet-runtime	Run performance job script	Log
2763485	dotnet-orleans	Test	Log
2763483	dotnet-performance	Send job to Helix (Windows)	Log
2763456	dotnet-runtime	Send job to Helix (Unix)	Log
1112094	dotnet/runtime	Send tests to Helix (Windows)	Log
1112093	dotnet/runtime	AOT compile CoreCLR tests	Log
2763398	dotnet-runtime	Run performance job script	Log
2763349	dotnet-runtime	Run performance job script	Log
1112022	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118165
1112018	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118241
1112015	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#118302
1111931	dotnet/runtime	Send tests to Helix (Unix)	Log
2763287	dotnet-performance	Send job to Helix (Windows)	Log
1111950	dotnet/roslyn	Test	Log	dotnet/roslyn#79751
1111940	dotnet/roslyn	Run Integration Tests	Log
1111938	dotnet/roslyn	Run Integration Tests	Log
1111939	dotnet/roslyn	Test	Log
2763146	dotnet-maui	Provision Android SDK - Common Packages	Log
2763118	dotnet-runtime	Send job to Helix (Windows)	Log
1111871	dotnet/roslyn	Run Integration Tests	Log
2763092	dotnet-runtime	Run performance job script	Log
1111846	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79562
1111635	dotnet/runtime	Send to Helix	Log
1111778	dotnet/roslyn	Run Integration Tests	Log
2763028	dotnet-performance	Send job to Helix (Windows)	Log
1111698	dotnet/runtime	Send to Helix	Log	dotnet/runtime#118256
1111732	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79746
2762999	dotnet-runtime	Run performance job script	Log
1111687	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79742
2762982	dotnet-dotnet	Checkout dotnet-dotnet@non-prod-pipeline to s	Log
1111646	dotnet/roslyn	Run Integration Tests	Log
2762908	dotnet-dotnet	Build (centOSStreamContainer)	Log
2762894	dotnet-performance	Send job to Helix (Windows)	Log
1111371	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79737
2762717	dotnet-runtime	Send job to Helix (Windows)	Log
2762733	dotnet-dotnet-docker	🛡 Guardian: CodeQL - TypeScript and JavaScript	Log
Displaying 100 of 1387 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
51	221	1387

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez