Skip to content

Improve and simplify the microbenchmarks CI setup #7613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Improve and simplify the microbenchmarks CI setup #7613

wants to merge 4 commits into from

Conversation

igoragoli
Copy link
Contributor

@igoragoli igoragoli commented Oct 3, 2025 •
edited
Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Test run on the CI: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-dotnet/-/jobs/1161521230

Other details

@igoragoli igoragoli marked this pull request as ready for review October 3, 2025 07:54
@igoragoli igoragoli requested a review from a team as a code owner October 3, 2025 07:54
@pr-commenter
Copy link

pr-commenter bot commented Oct 3, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7613 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.170
  • 1 benchmarks are slower, with geometric mean 2.416
  • 6 benchmarks have fewer allocations
  • 5 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.07 KB 6.02 KB -48 B -0.79%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.69 KB 5.64 KB -48 B -0.84%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.5μs 56.3ns 303ns 0 0 0 5.51 KB
master StartStopWithChild netcoreapp3.1 13.4μs 68.6ns 314ns 0 0 0 5.69 KB
master StartStopWithChild net472 22μs 94.1ns 352ns 0.891 0.223 0 6.07 KB
#7613 StartStopWithChild net6.0 10.6μs 57.8ns 365ns 0 0 0 5.49 KB
#7613 StartStopWithChild netcoreapp3.1 13.4μs 69.4ns 333ns 0 0 0 5.64 KB
#7613 StartStopWithChild net472 22.2μs 122ns 724ns 0.997 0.332 0.111 6.02 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 928μs 209ns 753ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.05ms 540ns 2.09μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.2ms 414ns 1.6μs 0 0 0 3.31 KB
#7613 WriteAndFlushEnrichedTraces net6.0 933μs 145ns 524ns 0 0 0 2.7 KB
#7613 WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 101ns 365ns 0 0 0 2.7 KB
#7613 WriteAndFlushEnrichedTraces net472 1.18ms 151ns 565ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 328μs 221ns 856ns 0 0 0 175.99 KB
master AllCycleSimpleBody netcoreapp3.1 500μs 1.83μs 7.08μs 0 0 0 178.17 KB
master AllCycleSimpleBody net472 444μs 104ns 388ns 30.2 0 0 200.21 KB
master AllCycleMoreComplexBody net6.0 343μs 1.23μs 4.76μs 0 0 0 179.5 KB
master AllCycleMoreComplexBody netcoreapp3.1 506μs 637ns 2.47μs 0 0 0 181.58 KB
master AllCycleMoreComplexBody net472 454μs 428ns 1.66μs 31.2 0 0 203.73 KB
master ObjectExtractorSimpleBody net6.0 327ns 1.58ns 6.89ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 396ns 2.21ns 14.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 297ns 0.0589ns 0.22ns 0.0434 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.45μs 11.6ns 45ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.86μs 27.3ns 106ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.65μs 7.89ns 30.6ns 0.601 0 0 3.8 KB
#7613 AllCycleSimpleBody net6.0 328μs 160ns 622ns 0 0 0 176 KB
#7613 AllCycleSimpleBody netcoreapp3.1 501μs 233ns 901ns 0 0 0 178.17 KB
#7613 AllCycleSimpleBody net472 445μs 87.3ns 338ns 31.2 0 0 200.21 KB
#7613 AllCycleMoreComplexBody net6.0 342μs 456ns 1.76μs 0 0 0 179.5 KB
#7613 AllCycleMoreComplexBody netcoreapp3.1 481μs 2.22μs 9.68μs 0 0 0 181.58 KB
#7613 AllCycleMoreComplexBody net472 452μs 198ns 767ns 31.2 0 0 203.73 KB
#7613 ObjectExtractorSimpleBody net6.0 335ns 0.107ns 0.414ns 0 0 0 280 B
#7613 ObjectExtractorSimpleBody netcoreapp3.1 413ns 0.315ns 1.22ns 0 0 0 272 B
#7613 ObjectExtractorSimpleBody net472 296ns 0.025ns 0.0967ns 0.0433 0 0 281 B
#7613 ObjectExtractorMoreComplexBody net6.0 6.26μs 31.6ns 138ns 0 0 0 3.78 KB
#7613 ObjectExtractorMoreComplexBody netcoreapp3.1 7.74μs 36.8ns 142ns 0 0 0 3.69 KB
#7613 ObjectExtractorMoreComplexBody net472 6.73μs 7.48ns 29ns 0.603 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 76.9μs 252ns 974ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 97μs 86.4ns 335ns 0 0 0 32.4 KB
master EncodeArgs net472 110μs 17ns 65.7ns 4.96 0 0 32.51 KB
master EncodeLegacyArgs net6.0 144μs 14.5ns 54.2ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 196μs 182ns 681ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 262μs 22.8ns 88.4ns 0 0 0 2.16 KB
#7613 EncodeArgs net6.0 77.8μs 87.3ns 338ns 0 0 0 32.4 KB
#7613 EncodeArgs netcoreapp3.1 95.9μs 287ns 1.04μs 0 0 0 32.4 KB
#7613 EncodeArgs net472 110μs 32.6ns 126ns 4.95 0 0 32.51 KB
#7613 EncodeLegacyArgs net6.0 143μs 64ns 222ns 0 0 0 2.15 KB
#7613 EncodeLegacyArgs netcoreapp3.1 201μs 136ns 526ns 0 0 0 2.14 KB
#7613 EncodeLegacyArgs net472 262μs 33.3ns 129ns 0 0 0 2.17 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #7613

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 2.416 298,311.31 720,603.42

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 395μs 121ns 451ns 0 0 0 4.56 KB
master RunWafRealisticBenchmark netcoreapp3.1 412μs 99.5ns 345ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 440μs 98.6ns 369ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 288μs 86ns 333ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 298μs 20.2ns 78.1ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 310μs 34.4ns 133ns 0 0 0 2.29 KB
#7613 RunWafRealisticBenchmark net6.0 394μs 54.1ns 210ns 0 0 0 4.56 KB
#7613 RunWafRealisticBenchmark netcoreapp3.1 412μs 233ns 904ns 0 0 0 4.48 KB
#7613 RunWafRealisticBenchmark net472 426μs 73.5ns 285ns 0 0 0 4.66 KB
#7613 RunWafRealisticBenchmarkWithAttack net6.0 292μs 65.3ns 253ns 0 0 0 2.24 KB
#7613 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 678μs 13.5μs 127μs 0 0 0 2.22 KB
#7613 RunWafRealisticBenchmarkWithAttack net472 308μs 18.7ns 67.3ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.6μs 66.5ns 249ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 72μs 54.9ns 205ns 0 0 0 17.42 KB
master SendRequest net472 0.00395ns 0.00183ns 0.0071ns 0 0 0 0 b
#7613 SendRequest net6.0 60.9μs 75ns 281ns 0 0 0 14.52 KB
#7613 SendRequest netcoreapp3.1 70.8μs 130ns 851ns 0 0 0 17.42 KB
#7613 SendRequest net472 0.0129ns 0.0037ns 0.0143ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7613

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 1.188 1,632,575.00 1,374,754.17
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1 1.167 1,920,338.75 1,645,471.25
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑net6.0 1.154 2,170,895.83 1,880,507.59

More allocations ⚠️ in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 2 B 5 B 3 B 150.00%

Fewer allocations 🎉 in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 8 B 4 B -4 B -50.00%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 2.17ms 987ns 3.69μs 0 0 0 640 KB
master OriginalCharSlice netcoreapp3.1 2.31ms 5.1μs 19.1μs 0 0 0 640 KB
master OriginalCharSlice net472 2.7ms 2.86μs 11.1μs 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.63ms 524ns 2.03μs 0 0 0 8 B
master OptimizedCharSlice netcoreapp3.1 1.92ms 535ns 2.07μs 0 0 0 1 B
master OptimizedCharSlice net472 2.02ms 2.17μs 8.39μs 0 0 0 73 B
master OptimizedCharSliceWithPool net6.0 916μs 43.1ns 167ns 0 0 0 2 B
master OptimizedCharSliceWithPool netcoreapp3.1 871μs 92.5ns 358ns 0 0 0 0 b
master OptimizedCharSliceWithPool net472 1.16ms 112ns 435ns 0 0 0 47 B
#7613 OriginalCharSlice net6.0 1.87ms 6.49μs 24.3μs 0 0 0 640 KB
#7613 OriginalCharSlice netcoreapp3.1 2.09ms 2.42μs 8.73μs 0 0 0 640 KB
#7613 OriginalCharSlice net472 2.79ms 6.19μs 24μs 100 0 0 641.95 KB
#7613 OptimizedCharSlice net6.0 1.37ms 287ns 1.07μs 0 0 0 4 B
#7613 OptimizedCharSlice netcoreapp3.1 1.65ms 533ns 2.07μs 0 0 0 1 B
#7613 OptimizedCharSlice net472 2ms 422ns 1.64μs 0 0 0 73 B
#7613 OptimizedCharSliceWithPool net6.0 863μs 65.3ns 253ns 0 0 0 5 B
#7613 OptimizedCharSliceWithPool netcoreapp3.1 820μs 41ns 159ns 0 0 0 0 b
#7613 OptimizedCharSliceWithPool net472 1.17ms 69.2ns 268ns 0 0 0 47 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 56.53 KB 55.93 KB -596 B -1.05%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 688μs 408ns 1.53μs 0 0 0 41.76 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 651μs 2.27μs 8.78μs 0 0 0 41.98 KB
master WriteAndFlushEnrichedTraces net472 902μs 3.45μs 13.3μs 8.33 0 0 56.53 KB
#7613 WriteAndFlushEnrichedTraces net6.0 695μs 1.4μs 5.41μs 0 0 0 41.77 KB
#7613 WriteAndFlushEnrichedTraces netcoreapp3.1 667μs 2.57μs 8.91μs 0 0 0 41.96 KB
#7613 WriteAndFlushEnrichedTraces net472 866μs 3.07μs 11.5μs 4.46 0 0 55.93 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.95μs 8.65ns 32.4ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.61μs 8.15ns 31.5ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.77μs 6.51ns 25.2ns 0.153 0.0139 0 987 B
#7613 ExecuteNonQuery net6.0 1.93μs 9.63ns 40.8ns 0 0 0 1.02 KB
#7613 ExecuteNonQuery netcoreapp3.1 2.63μs 9.15ns 35.4ns 0 0 0 1.02 KB
#7613 ExecuteNonQuery net472 2.75μs 4.89ns 18.9ns 0.151 0.0137 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.74μs 4.27ns 16.5ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.17μs 11.2ns 52.7ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.43μs 3.2ns 12.4ns 0.155 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.84μs 0.841ns 3.26ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.43μs 10.9ns 42.1ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.75μs 1.91ns 7.41ns 0.168 0 0 1.1 KB
#7613 CallElasticsearch net6.0 1.72μs 8.43ns 36.8ns 0 0 0 1.03 KB
#7613 CallElasticsearch netcoreapp3.1 2.27μs 11.4ns 48.2ns 0 0 0 1.03 KB
#7613 CallElasticsearch net472 3.44μs 3.48ns 13ns 0.156 0 0 1.04 KB
#7613 CallElasticsearchAsync net6.0 1.81μs 9.12ns 38.7ns 0 0 0 1.01 KB
#7613 CallElasticsearchAsync netcoreapp3.1 2.39μs 10.4ns 39ns 0 0 0 1.08 KB
#7613 CallElasticsearchAsync net472 3.78μs 5.38ns 20.8ns 0.169 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.9μs 7.99ns 30.9ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.37μs 2.84ns 11ns 0 0 0 952 B
master ExecuteAsync net472 2.72μs 2.77ns 10.7ns 0.136 0 0 915 B
#7613 ExecuteAsync net6.0 1.84μs 9.24ns 39.2ns 0 0 0 952 B
#7613 ExecuteAsync netcoreapp3.1 2.31μs 10ns 36.1ns 0 0 0 952 B
#7613 ExecuteAsync net472 2.58μs 2.68ns 10.4ns 0.141 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.95μs 16.3ns 63ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.5μs 12.9ns 49.9ns 0 0 0 2.9 KB
master SendAsync net472 12.2μs 8.18ns 30.6ns 0.488 0 0 3.18 KB
#7613 SendAsync net6.0 6.91μs 13.3ns 51.4ns 0 0 0 2.36 KB
#7613 SendAsync netcoreapp3.1 8.15μs 28.5ns 111ns 0 0 0 2.9 KB
#7613 SendAsync net472 11.9μs 11ns 42.8ns 0.474 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 65.54 KB 8.19 KB 14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 257.76 KB 275.78 KB 18.02 KB 6.99%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 259.54 KB 276.63 KB 17.09 KB 6.58%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 278.53 KB 286.72 KB 8.19 KB 2.94%

Fewer allocations 🎉 in #7613

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 44.11 KB 43.53 KB -584 B -1.32%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 86.74 KB 42.94 KB -43.81 KB -50.50%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 47.3μs 692ns 6.78μs 0 0 0 44.11 KB
master StringConcatBenchmark netcoreapp3.1 49.4μs 282ns 1.94μs 0 0 0 86.74 KB
master StringConcatBenchmark net472 57.7μs 190ns 710ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 463μs 1.68μs 6.05μs 0 0 0 259.54 KB
master StringConcatAspectBenchmark netcoreapp3.1 493μs 2.2μs 8.5μs 0 0 0 257.76 KB
master StringConcatAspectBenchmark net472 396μs 1.25μs 4.66μs 0 0 0 278.53 KB
#7613 StringConcatBenchmark net6.0 45.2μs 264ns 2.17μs 0 0 0 43.53 KB
#7613 StringConcatBenchmark netcoreapp3.1 50.3μs 297ns 2.62μs 0 0 0 42.94 KB
#7613 StringConcatBenchmark net472 56.9μs 258ns 1.18μs 0 0 0 65.54 KB
#7613 StringConcatAspectBenchmark net6.0 497μs 1.8μs 6.23μs 0 0 0 276.63 KB
#7613 StringConcatAspectBenchmark netcoreapp3.1 456μs 5.43μs 52.9μs 0 0 0 275.78 KB
#7613 StringConcatAspectBenchmark net472 396μs 1.59μs 6.36μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.52μs 12.8ns 62.6ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.35μs 17.4ns 85.4ns 0 0 0 1.7 KB
master EnrichedLog net472 3.77μs 6.44ns 24.9ns 0.244 0 0 1.64 KB
#7613 EnrichedLog net6.0 2.54μs 1.21ns 4.21ns 0 0 0 1.7 KB
#7613 EnrichedLog netcoreapp3.1 3.49μs 14.8ns 57.3ns 0 0 0 1.7 KB
#7613 EnrichedLog net472 3.69μs 5.1ns 19.8ns 0.257 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 124μs 83.6ns 313ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 127μs 169ns 653ns 0 0 0 4.31 KB
master EnrichedLog net472 168μs 221ns 798ns 0 0 0 4.52 KB
#7613 EnrichedLog net6.0 123μs 258ns 966ns 0 0 0 4.31 KB
#7613 EnrichedLog netcoreapp3.1 128μs 61.7ns 239ns 0 0 0 4.31 KB
#7613 EnrichedLog net472 168μs 76.3ns 286ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.87μs 13.3ns 49.7ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.57μs 18.5ns 71.7ns 0 0 0 2.26 KB
master EnrichedLog net472 7.17μs 6.31ns 24.4ns 0.324 0 0 2.08 KB
#7613 EnrichedLog net6.0 4.83μs 9.29ns 34.8ns 0 0 0 2.26 KB
#7613 EnrichedLog netcoreapp3.1 6.84μs 6.58ns 24.6ns 0 0 0 2.26 KB
#7613 EnrichedLog net472 7.4μs 14ns 54.3ns 0.296 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.98μs 9.62ns 41.9ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.59μs 12.2ns 50.2ns 0 0 0 1.2 KB
master SendReceive net472 3.13μs 1.94ns 7.52ns 0.187 0 0 1.2 KB
#7613 SendReceive net6.0 2μs 10.3ns 49.5ns 0 0 0 1.2 KB
#7613 SendReceive netcoreapp3.1 2.59μs 6.94ns 26.9ns 0 0 0 1.2 KB
#7613 SendReceive net472 3.15μs 1.37ns 5.3ns 0.19 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.12μs 6.52ns 25.3ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.28μs 13ns 48.8ns 0 0 0 1.63 KB
master EnrichedLog net472 6.2μs 6ns 23.3ns 0.311 0 0 2.03 KB
#7613 EnrichedLog net6.0 4.1μs 1.44ns 5.4ns 0 0 0 1.58 KB
#7613 EnrichedLog netcoreapp3.1 5.43μs 7ns 27.1ns 0 0 0 1.63 KB
#7613 EnrichedLog net472 6.41μs 6.67ns 24ns 0.29 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 761ns 3.3ns 12.8ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 917ns 1.12ns 4.33ns 0 0 0 576 B
master StartFinishSpan net472 872ns 0.0793ns 0.297ns 0.0917 0 0 578 B
master StartFinishScope net6.0 880ns 0.519ns 2.01ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.19μs 2.88ns 11.1ns 0 0 0 696 B
master StartFinishScope net472 1.03μs 0.37ns 1.43ns 0.104 0 0 658 B
#7613 StartFinishSpan net6.0 746ns 3.55ns 14.6ns 0 0 0 576 B
#7613 StartFinishSpan netcoreapp3.1 920ns 4.26ns 17.6ns 0 0 0 576 B
#7613 StartFinishSpan net472 914ns 0.226ns 0.783ns 0.0911 0 0 578 B
#7613 StartFinishScope net6.0 902ns 4.39ns 19.6ns 0 0 0 696 B
#7613 StartFinishScope netcoreapp3.1 1.13μs 6.2ns 34.5ns 0 0 0 696 B
#7613 StartFinishScope net472 1.09μs 5.15ns 19.9ns 0.102 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 2.57ns 9.96ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.41μs 7.28ns 34.2ns 0 0 0 696 B
master RunOnMethodBegin net472 1.4μs 0.226ns 0.875ns 0.0982 0 0 658 B
#7613 RunOnMethodBegin net6.0 1.07μs 5.5ns 26.9ns 0 0 0 696 B
#7613 RunOnMethodBegin netcoreapp3.1 1.38μs 5.09ns 19.7ns 0 0 0 696 B
#7613 RunOnMethodBegin net472 1.47μs 1.14ns 4.26ns 0.103 0 0 658 B

@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Oct 3, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (81ms)  : 77, 84
     .   : milestone, 81,
    master - mean (80ms)  : 77, 83
     .   : milestone, 80,

    section Baseline
    This PR (7613) - mean (76ms)  : 73, 80
     .   : milestone, 76,
    master - mean (76ms)  : 72, 79
     .   : milestone, 76,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (1,118ms)  : 1079, 1157
     .   : milestone, 1118,
    master - mean (1,124ms)  : 1051, 1196
     .   : milestone, 1124,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (120ms)  : 115, 126
     .   : milestone, 120,
    master - mean (120ms)  : 114, 126
     .   : milestone, 120,

    section Baseline
    This PR (7613) - mean (119ms)  : 114, 124
     .   : milestone, 119,
    master - mean (119ms)  : 113, 125
     .   : milestone, 119,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (803ms)  : 775, 831
     .   : milestone, 803,
    master - mean (797ms)  : 768, 826
     .   : milestone, 797,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (114ms)  : 109, 119
     .   : milestone, 114,
    master - mean (113ms)  : 109, 118
     .   : milestone, 113,

    section Baseline
    This PR (7613) - mean (113ms)  : 106, 119
     .   : milestone, 113,
    master - mean (112ms)  : 106, 119
     .   : milestone, 112,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (840ms)  : 788, 891
     .   : milestone, 840,
    master - mean (839ms)  : 779, 898
     .   : milestone, 839,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (105ms)  : 101, 110
     .   : milestone, 105,
    master - mean (105ms)  : 100, 110
     .   : milestone, 105,

    section Baseline
    This PR (7613) - mean (104ms)  : 98, 110
     .   : milestone, 104,
    master - mean (103ms)  : 98, 108
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (726ms)  : 699, 752
     .   : milestone, 726,
    master - mean (720ms)  : 690, 750
     .   : milestone, 720,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (202ms)  : 193, 211
     .   : milestone, 202,
    master - mean (201ms)  : 198, 204
     .   : milestone, 201,

    section Baseline
    This PR (7613) - mean (201ms)  : 189, 212
     .   : milestone, 201,
    master - mean (197ms)  : 193, 200
     .   : milestone, 197,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (1,235ms)  : 1153, 1317
     .   : milestone, 1235,
    master - mean (1,179ms)  : 1118, 1240
     .   : milestone, 1179,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (299ms)  : crit, 290, 307
     .   : crit, milestone, 299,
    master - mean (283ms)  : 278, 288
     .   : milestone, 283,

    section Baseline
    This PR (7613) - mean (294ms)  : 283, 305
     .   : milestone, 294,
    master - mean (283ms)  : 276, 291
     .   : milestone, 283,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (989ms)  : 948, 1029
     .   : milestone, 989,
    master - mean (951ms)  : 901, 1001
     .   : milestone, 951,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (290ms)  : 281, 299
     .   : milestone, 290,
    master - mean (288ms)  : 281, 296
     .   : milestone, 288,

    section Baseline
    This PR (7613) - mean (294ms)  : 279, 309
     .   : milestone, 294,
    master - mean (288ms)  : 283, 293
     .   : milestone, 288,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (1,020ms)  : 977, 1062
     .   : milestone, 1020,
    master - mean (1,020ms)  : 967, 1073
     .   : milestone, 1020,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7613) - mean (280ms)  : 273, 286
     .   : milestone, 280,
    master - mean (283ms)  : 273, 292
     .   : milestone, 283,

    section Baseline
    This PR (7613) - mean (278ms)  : 272, 283
     .   : milestone, 278,
    master - mean (281ms)  : 270, 292
     .   : milestone, 281,

    section CallTarget+Inlining+NGEN
    This PR (7613) - mean (888ms)  : 835, 941
     .   : milestone, 888,
    master - mean (873ms)  : 840, 906
     .   : milestone, 873,
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@igoragoli @NachoEchevarria