-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | ViperUbuntu |
Baseline | 201b515d1e646f689efbee52c3585e85d4c3cbf4 |
Compare | 1390e22bd17e06147dfd382c733b591aa9fb8d28 |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Regressions in PerfLabTests.DelegatePerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
2.75 ms | 3.15 ms | 1.15 | 0.16 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.DelegatePerf*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'PerfLabTests.DelegatePerf*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'PerfLabTests.DelegatePerf*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'PerfLabTests.DelegatePerf*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
PerfLabTests.DelegatePerf.DelegateInvoke
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | ViperUbuntu |
Baseline | 201b515d1e646f689efbee52c3585e85d4c3cbf4 |
Compare | 1390e22bd17e06147dfd382c733b591aa9fb8d28 |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
113.75 ns | 132.43 ns | 1.16 | 0.15 | False | ||||
98.07 ns | 106.31 ns | 1.08 | 0.13 | False | ||||
123.62 ns | 144.86 ns | 1.17 | 0.16 | False | ||||
100.96 ns | 108.62 ns | 1.08 | 0.10 | False | ||||
92.63 ns | 115.33 ns | 1.25 | 0.18 | False | ||||
35.21 ns | 41.58 ns | 1.18 | 0.23 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid
ETL Files
Histogram
JIT Disasms
Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf
ETL Files
Histogram
JIT Disasms
Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid
ETL Files
Histogram
JIT Disasms
Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode
ETL Files
Histogram
JIT Disasms
Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String
ETL Files
Histogram
JIT Disasms
Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubSegment
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | ViperUbuntu |
Baseline | 201b515d1e646f689efbee52c3585e85d4c3cbf4 |
Compare | 1390e22bd17e06147dfd382c733b591aa9fb8d28 |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Regressions in System.Memory.Span<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
142.95 ns | 163.24 ns | 1.14 | 0.32 | False | ||||
571.91 ns | 750.20 ns | 1.31 | 0.57 | False | ||||
121.85 ns | 146.33 ns | 1.20 | 0.16 | False | ||||
1.44 μs | 1.84 μs | 1.28 | 0.45 | False | ||||
877.97 ns | 1.05 μs | 1.20 | 0.42 | False | ||||
66.31 ns | 74.09 ns | 1.12 | 0.34 | False | ||||
103.67 ns | 120.10 ns | 1.16 | 0.23 | False | ||||
813.65 ns | 923.34 ns | 1.13 | 0.48 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Int32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Int32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Int32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Int32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
System.Memory.Span<Int32>.SequenceEqual(Size: 33)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.Fill(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.EndsWith(Size: 33)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.SequenceEqual(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.EndsWith(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.Fill(Size: 33)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.StartsWith(Size: 33)
ETL Files
Histogram
JIT Disasms
System.Memory.Span<Int32>.StartsWith(Size: 512)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | ViperUbuntu |
Baseline | 201b515d1e646f689efbee52c3585e85d4c3cbf4 |
Compare | 1390e22bd17e06147dfd382c733b591aa9fb8d28 |
Diff | Diff |
Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono |
Regressions in BenchmarksGame.PiDigits_3
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
5.98 secs | 6.30 secs | 1.05 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BenchmarksGame.PiDigits_3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'BenchmarksGame.PiDigits_3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BenchmarksGame.PiDigits_3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'BenchmarksGame.PiDigits_3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
BenchmarksGame.PiDigits_3.RunBench(n: 3000, expected: "8649423196\t:3000")
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository