Skip to content

Supporting changes to implement python pipeline for performance runs on runtime repo #4156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Sep 16, 2024
+491 −287
Merged

Supporting changes to implement python pipeline for performance runs on runtime repo #4156

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
675409e
Make changes to python pipeline for runtime repo
caaavik-msft Mar 7, 2024
a99de98
More fixes
caaavik-msft Mar 8, 2024
e0f9132
Clean up some parts
caaavik-msft Mar 9, 2024
6f5cc5a
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Apr 12, 2024
bfc50f1
Add latest changes from runtime repo
caaavik-msft Apr 12, 2024
5b9f5d7
Fix wasm bundle path and add arg support for local commit time
caaavik-msft Apr 16, 2024
0e43842
Fix local commit time code
caaavik-msft Apr 16, 2024
4df7faf
Only set core_root for coreclr runs
caaavik-msft Apr 16, 2024
75c1a48
Fix ios hello world copying
caaavik-msft Apr 16, 2024
d4b116f
Fix wasm copying and prereq check on linux
caaavik-msft Apr 16, 2024
fd90221
Fix wasm args quoting and PreparePayloadWorkItems usage
caaavik-msft Apr 16, 2024
e2ea387
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Apr 16, 2024
d158e8b
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Jul 20, 2024
6c68491
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Jul 20, 2024
da1ad30
Use new browser versions path
caaavik-msft Jul 22, 2024
5f16a3a
Fix prerequisites script building
caaavik-msft Jul 22, 2024
3f393bc
Fix wasm args building
caaavik-msft Jul 22, 2024
fa5345c
Fix directory creation and prevent pip reinstalls
caaavik-msft Jul 22, 2024
f44f241
Fix trailing space on wasm args
caaavik-msft Jul 22, 2024
017204e
More various fixes
caaavik-msft Jul 22, 2024
cd5fdb2
Fix file copy into root
caaavik-msft Jul 22, 2024
8bb2497
Fix FailOnTestFailure value
caaavik-msft Jul 23, 2024
d5cdb7a
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Aug 2, 2024
88df1c1
Merge branch 'main' into caaavik/python-pipeline-for-runtime
caaavik-msft Sep 6, 2024
2fc8c94
Include recent changes from runtime repo
caaavik-msft Sep 6, 2024
41ec18e
Only build startup and SOD for runtime scenarios
caaavik-msft Sep 7, 2024
a8b43bb
Fix infinite loop in zip creation
caaavik-msft Sep 7, 2024
d79455a
Fix some directory issues
caaavik-msft Sep 7, 2024
3ea8dd3
Fix some copytree logic
caaavik-msft Sep 7, 2024
d9dcf41
Copy ios apps into correct dir
caaavik-msft Sep 8, 2024
98f2a60
Fix iosSampleApp copying + add better logging
caaavik-msft Sep 8, 2024
9e69d14
Fix PDN download and error code
caaavik-msft Sep 9, 2024
38333a3
Address PR comments
caaavik-msft Sep 9, 2024
dff7fec
Fix incorrect tfm condition check
caaavik-msft Sep 11, 2024
a4a0d97
Address PR comments
caaavik-msft Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 54 additions & 34 deletions eng/performance/helix.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,56 +11,76 @@
<PartitionCount Condition="'$(PartitionCount)' == ''">15</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="Partition0" Index="0" Condition="$(PartitionCount) &gt; 0" />
<Partition Include="Partition1" Index="1" Condition="$(PartitionCount) &gt; 1" />
<Partition Include="Partition2" Index="2" Condition="$(PartitionCount) &gt; 2" />
<Partition Include="Partition3" Index="3" Condition="$(PartitionCount) &gt; 3" />
<Partition Include="Partition4" Index="4" Condition="$(PartitionCount) &gt; 4" />
<Partition Include="Partition5" Index="5" Condition="$(PartitionCount) &gt; 5" />
<Partition Include="Partition6" Index="6" Condition="$(PartitionCount) &gt; 6" />
<Partition Include="Partition7" Index="7" Condition="$(PartitionCount) &gt; 7" />
<Partition Include="Partition8" Index="8" Condition="$(PartitionCount) &gt; 8" />
<Partition Include="Partition9" Index="9" Condition="$(PartitionCount) &gt; 9" />
<Partition Include="Partition10" Index="10" Condition="$(PartitionCount) &gt; 10" />
<Partition Include="Partition11" Index="11" Condition="$(PartitionCount) &gt; 11" />
<Partition Include="Partition12" Index="12" Condition="$(PartitionCount) &gt; 12" />
<Partition Include="Partition13" Index="13" Condition="$(PartitionCount) &gt; 13" />
<Partition Include="Partition14" Index="14" Condition="$(PartitionCount) &gt; 14" />
<Partition Include="Partition15" Index="15" Condition="$(PartitionCount) &gt; 15" />
<Partition Include="Partition16" Index="16" Condition="$(PartitionCount) &gt; 16" />
<Partition Include="Partition17" Index="17" Condition="$(PartitionCount) &gt; 17" />
<Partition Include="Partition18" Index="18" Condition="$(PartitionCount) &gt; 18" />
<Partition Include="Partition19" Index="19" Condition="$(PartitionCount) &gt; 19" />
<Partition Include="Partition20" Index="20" Condition="$(PartitionCount) &gt; 20" />
<Partition Include="Partition21" Index="21" Condition="$(PartitionCount) &gt; 21" />
<Partition Include="Partition22" Index="22" Condition="$(PartitionCount) &gt; 22" />
<Partition Include="Partition23" Index="23" Condition="$(PartitionCount) &gt; 23" />
<Partition Include="Partition24" Index="24" Condition="$(PartitionCount) &gt; 24" />
<Partition Include="Partition25" Index="25" Condition="$(PartitionCount) &gt; 25" />
<Partition Include="Partition26" Index="26" Condition="$(PartitionCount) &gt; 26" />
<Partition Include="Partition27" Index="27" Condition="$(PartitionCount) &gt; 27" />
<Partition Include="Partition28" Index="28" Condition="$(PartitionCount) &gt; 28" />
<Partition Include="Partition29" Index="29" Condition="$(PartitionCount) &gt; 29" />
<Partition Include="$(BuildConfig).Partition0" Index="0" Condition="$(PartitionCount) &gt; 0" />
</ItemGroup>

<ItemGroup Condition="'$(OnlySanityCheck)' != 'true'">
<Partition Include="$(BuildConfig).Partition1" Index="1" Condition="$(PartitionCount) &gt; 1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" Condition="$(PartitionCount) &gt; 2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" Condition="$(PartitionCount) &gt; 3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" Condition="$(PartitionCount) &gt; 4" />
<Partition Include="$(BuildConfig).Partition5" Index="5" Condition="$(PartitionCount) &gt; 5" />
<Partition Include="$(BuildConfig).Partition6" Index="6" Condition="$(PartitionCount) &gt; 6" />
<Partition Include="$(BuildConfig).Partition7" Index="7" Condition="$(PartitionCount) &gt; 7" />
<Partition Include="$(BuildConfig).Partition8" Index="8" Condition="$(PartitionCount) &gt; 8" />
<Partition Include="$(BuildConfig).Partition9" Index="9" Condition="$(PartitionCount) &gt; 9" />
<Partition Include="$(BuildConfig).Partition10" Index="10" Condition="$(PartitionCount) &gt; 10" />
<Partition Include="$(BuildConfig).Partition11" Index="11" Condition="$(PartitionCount) &gt; 11" />
<Partition Include="$(BuildConfig).Partition12" Index="12" Condition="$(PartitionCount) &gt; 12" />
<Partition Include="$(BuildConfig).Partition13" Index="13" Condition="$(PartitionCount) &gt; 13" />
<Partition Include="$(BuildConfig).Partition14" Index="14" Condition="$(PartitionCount) &gt; 14" />
<Partition Include="$(BuildConfig).Partition15" Index="15" Condition="$(PartitionCount) &gt; 15" />
<Partition Include="$(BuildConfig).Partition16" Index="16" Condition="$(PartitionCount) &gt; 16" />
<Partition Include="$(BuildConfig).Partition17" Index="17" Condition="$(PartitionCount) &gt; 17" />
<Partition Include="$(BuildConfig).Partition18" Index="18" Condition="$(PartitionCount) &gt; 18" />
<Partition Include="$(BuildConfig).Partition19" Index="19" Condition="$(PartitionCount) &gt; 19" />
<Partition Include="$(BuildConfig).Partition20" Index="20" Condition="$(PartitionCount) &gt; 20" />
<Partition Include="$(BuildConfig).Partition21" Index="21" Condition="$(PartitionCount) &gt; 21" />
<Partition Include="$(BuildConfig).Partition22" Index="22" Condition="$(PartitionCount) &gt; 22" />
<Partition Include="$(BuildConfig).Partition23" Index="23" Condition="$(PartitionCount) &gt; 23" />
<Partition Include="$(BuildConfig).Partition24" Index="24" Condition="$(PartitionCount) &gt; 24" />
<Partition Include="$(BuildConfig).Partition25" Index="25" Condition="$(PartitionCount) &gt; 25" />
<Partition Include="$(BuildConfig).Partition26" Index="26" Condition="$(PartitionCount) &gt; 26" />
<Partition Include="$(BuildConfig).Partition27" Index="27" Condition="$(PartitionCount) &gt; 27" />
<Partition Include="$(BuildConfig).Partition28" Index="28" Condition="$(PartitionCount) &gt; 28" />
<Partition Include="$(BuildConfig).Partition29" Index="29" Condition="$(PartitionCount) &gt; 29" />
</ItemGroup>

<!--
Partition the MicroBenchmarks project, but nothing else
TODO: Support Compare runs
This is achieved by doing a baseline corerun in a precommand, then running ResultsComparer in a postcommand
TODO: Check PERF_PREREQS_INSTALL_FAILED before running commands on non-windows
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --partition=%(HelixWorkItem.Index) --bdn-arguments="$(BenchmarkDotNetArguments) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<PreCommands Condition="'$(Compare)' == 'true'">$(BaselineWorkItemCommand) --partition=%(HelixWorkItem.Index) --bdn-arguments="$(BaselineBenchmarkDotNetArguments) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</PreCommands>
<Command Condition="'$(TargetsWindows)' == 'true'">$(WorkItemCommand) --partition=%(HelixWorkItem.Index) --bdn-arguments="$(BenchmarkDotNetArguments) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<Command Condition="'$(TargetsWindows)' != 'true'">
if [ "x$PERF_PREREQS_INSTALL_FAILED" = "x1" ]; then
echo "\n\n** Error: Failed to install prerequisites **\n\n"; (exit 1);
else
$(WorkItemCommand) --partition=%(HelixWorkItem.Index) --bdn-arguments="$(BenchmarkDotNetArguments) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)";
fi
</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(CompareCommand)</PostCommands>
<Timeout>$(WorkItemTimeout)</Timeout>
<DownloadFilesFromResults Condition="'$(DownloadFilesFromHelix)' == 'true'">Partition%(HelixWorkItem.Index)-combined-perf-lab-report.json</DownloadFilesFromResults>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="WorkItem">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --bdn-arguments="$(BenchmarkDotNetArguments)"</Command>
<PreCommands Condition="'$(Compare)' == 'true'">$(BaselineWorkItemCommand) --bdn-arguments="$(BaselineBenchmarkDotNetArguments)"</PreCommands>
<Command Condition="'$(TargetsWindows)' == 'true'">$(WorkItemCommand) --bdn-arguments="$(BenchmarkDotNetArguments)"</Command>
<Command Condition="'$(TargetsWindows)' != 'true'">
if [ "x$PERF_PREREQS_INSTALL_FAILED" = "x1" ]; then
echo "\n\n** Error: Failed to install prerequisites **\n\n"; (exit 1);
else
$(WorkItemCommand) --bdn-arguments="$(BenchmarkDotNetArguments)";
fi
</Command>
<PostCommands Condition="'$(Compare)' == 'true'">$(CompareCommand)</PostCommands>
<Timeout>$(WorkItemTimeout)</Timeout>
<DownloadFilesFromResults Condition="'$(DownloadFilesFromHelix)' == 'true'">combined-perf-lab-report.json</DownloadFilesFromResults>
</HelixWorkItem>
Expand Down
Loading