fix(di): get pending probes as a copy #14608
Conversation
P403n1x87
added
the
Dynamic Instrumentation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 269 ± 3 ms. The average import time from base is: 271 ± 3 ms. The import time difference between this PR and base is: -1.7 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
When retrieving the list of pending probes, we make the registry return a copy of the collection to avoid mutation during iteration. In the worst case, Python does not warn that the list has been mutated while iterating over it, causing the iteration loop to end prematurely without any sings of errors.
P403n1x87
force-pushed
the
fix/di-copy-pending-probes
branch
from
3ee45c1 to
963e6e0
Compare
Performance SLOsComparing candidate fix/di-copy-pending-probes (1cc2d69) with baseline main (d2812f2) 📈 Performance Regressions (2 suites)📈 iast_aspects - 40/40✅ re_expand_aspectTime: ✅ 32.164µs (SLO: <40.000µs 📉 -19.6%) vs baseline: +0.6% Memory: ✅ 36.903MB (SLO: <39.000MB -5.4%) vs baseline: +3.8% ✅ re_expand_noaspectTime: ✅ 28.279µs (SLO: <40.000µs 📉 -29.3%) vs baseline: -1.2% Memory: ✅ 37.297MB (SLO: <39.000MB -4.4%) vs baseline: +4.9% ✅ re_findall_aspectTime: ✅ 2.905µs (SLO: <10.000µs 📉 -70.9%) vs baseline: +0.4% Memory: ✅ 37.198MB (SLO: <39.000MB -4.6%) vs baseline: +4.6% ✅ re_findall_noaspectTime: ✅ 1.423µs (SLO: <10.000µs 📉 -85.8%) vs baseline: +1.4% Memory: ✅ 37.120MB (SLO: <39.000MB -4.8%) vs baseline: +4.6% ✅ re_finditer_aspectTime: ✅ 4.441µs (SLO: <10.000µs 📉 -55.6%) vs baseline: -0.7% Memory: ✅ 37.277MB (SLO: <39.000MB -4.4%) vs baseline: +4.7% ✅ re_finditer_noaspectTime: ✅ 1.414µs (SLO: <10.000µs 📉 -85.9%) vs baseline: ~same Memory: ✅ 37.238MB (SLO: <39.000MB -4.5%) vs baseline: +4.7% ✅ re_fullmatch_aspectTime: ✅ 2.760µs (SLO: <10.000µs 📉 -72.4%) vs baseline: +3.7% Memory: ✅ 37.297MB (SLO: <39.000MB -4.4%) vs baseline: +4.8% ✅ re_fullmatch_noaspectTime: ✅ 1.284µs (SLO: <10.000µs 📉 -87.2%) vs baseline: -0.6% Memory: ✅ 37.159MB (SLO: <39.000MB -4.7%) vs baseline: +4.4% ✅ re_group_aspectTime: ✅ 3.051µs (SLO: <10.000µs 📉 -69.5%) vs baseline: +3.6% Memory: ✅ 36.884MB (SLO: <39.000MB -5.4%) vs baseline: +3.8% ✅ re_group_noaspectTime: ✅ 1.599µs (SLO: <10.000µs 📉 -84.0%) vs baseline: -1.0% Memory: ✅ 36.864MB (SLO: <39.000MB -5.5%) vs baseline: +3.7% ✅ re_groups_aspectTime: ✅ 3.466µs (SLO: <10.000µs 📉 -65.3%) vs baseline: 📈 +12.8% Memory: ✅ 36.884MB (SLO: <39.000MB -5.4%) vs baseline: +3.6% ✅ re_groups_noaspectTime: ✅ 1.695µs (SLO: <10.000µs 📉 -83.0%) vs baseline: ~same Memory: ✅ 37.198MB (SLO: <39.000MB -4.6%) vs baseline: +4.5% ✅ re_match_aspectTime: ✅ 3.079µs (SLO: <10.000µs 📉 -69.2%) vs baseline: 📈 +14.2% Memory: ✅ 36.903MB (SLO: <39.000MB -5.4%) vs baseline: +3.8% ✅ re_match_noaspectTime: ✅ 1.309µs (SLO: <10.000µs 📉 -86.9%) vs baseline: ~same Memory: ✅ 37.238MB (SLO: <39.000MB -4.5%) vs baseline: +5.0% ✅ re_search_aspectTime: ✅ 2.537µs (SLO: <10.000µs 📉 -74.6%) vs baseline: -0.4% Memory: ✅ 36.844MB (SLO: <39.000MB -5.5%) vs baseline: +3.8% ✅ re_search_noaspectTime: ✅ 1.200µs (SLO: <10.000µs 📉 -88.0%) vs baseline: -0.5% Memory: ✅ 37.218MB (SLO: <39.000MB -4.6%) vs baseline: +4.6% ✅ re_sub_aspectTime: ✅ 3.453µs (SLO: <10.000µs 📉 -65.5%) vs baseline: +0.4% Memory: ✅ 36.844MB (SLO: <39.000MB -5.5%) vs baseline: +3.6% ✅ re_sub_noaspectTime: ✅ 1.555µs (SLO: <10.000µs 📉 -84.4%) vs baseline: +1.6% Memory: ✅ 37.297MB (SLO: <39.000MB -4.4%) vs baseline: +5.2% ✅ re_subn_aspectTime: ✅ 3.688µs (SLO: <10.000µs 📉 -63.1%) vs baseline: +1.1% Memory: ✅ 36.923MB (SLO: <39.000MB -5.3%) vs baseline: +4.0% ✅ re_subn_noaspectTime: ✅ 1.626µs (SLO: <10.000µs 📉 -83.7%) vs baseline: +1.9% Memory: ✅ 36.884MB (SLO: <39.000MB -5.4%) vs baseline: +3.8% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.171µs (SLO: <20.000µs 📉 -84.1%) vs baseline: -1.0% Memory: ✅ 31.398MB (SLO: <34.000MB -7.7%) vs baseline: +4.9% ✅ 1-count-metrics-100-timesTime: ✅ 217.101µs (SLO: <250.000µs 📉 -13.2%) vs baseline: ~same Memory: ✅ 31.339MB (SLO: <34.000MB -7.8%) vs baseline: +5.0% ✅ 1-distribution-metric-1-timesTime: ✅ 3.421µs (SLO: <20.000µs 📉 -82.9%) vs baseline: 📈 +13.6% Memory: ✅ 31.339MB (SLO: <34.000MB -7.8%) vs baseline: +4.9% ✅ 1-distribution-metrics-100-timesTime: ✅ 199.173µs (SLO: <220.000µs -9.5%) vs baseline: +2.4% Memory: ✅ 31.280MB (SLO: <34.000MB -8.0%) vs baseline: +4.6% ✅ 1-gauge-metric-1-timesTime: ✅ 2.138µs (SLO: <20.000µs 📉 -89.3%) vs baseline: +1.1% Memory: ✅ 31.359MB (SLO: <34.000MB -7.8%) vs baseline: +5.2% ✅ 1-gauge-metrics-100-timesTime: ✅ 124.360µs (SLO: <150.000µs 📉 -17.1%) vs baseline: -0.7% Memory: ✅ 31.379MB (SLO: <34.000MB -7.7%) vs baseline: +4.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.542µs (SLO: <20.000µs 📉 -82.3%) vs baseline: +9.4% Memory: ✅ 31.280MB (SLO: <34.000MB -8.0%) vs baseline: +4.8% ✅ 1-rate-metrics-100-timesTime: ✅ 216.546µs (SLO: <250.000µs 📉 -13.4%) vs baseline: -0.1% Memory: ✅ 31.339MB (SLO: <34.000MB -7.8%) vs baseline: +5.1% ✅ 100-count-metrics-100-timesTime: ✅ 22.437ms (SLO: <23.500ms -4.5%) vs baseline: +1.9% Memory: ✅ 31.300MB (SLO: <34.000MB -7.9%) vs baseline: +4.8% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.153ms (SLO: <2.250ms -4.3%) vs baseline: +5.8% Memory: ✅ 31.320MB (SLO: <34.000MB -7.9%) vs baseline: +4.8% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.277ms (SLO: <1.550ms 📉 -17.6%) vs baseline: -0.8% Memory: ✅ 31.320MB (SLO: <34.000MB -7.9%) vs baseline: +5.1% ✅ 100-rate-metrics-100-timesTime: ✅ 2.287ms (SLO: <2.550ms 📉 -10.3%) vs baseline: +2.2% Memory: ✅ 31.300MB (SLO: <34.000MB -7.9%) vs baseline: +4.7% ✅ flush-1-metricTime: ✅ 4.229µs (SLO: <20.000µs 📉 -78.9%) vs baseline: +0.5% Memory: ✅ 31.575MB (SLO: <34.000MB -7.1%) vs baseline: +4.4% ✅ flush-100-metricsTime: ✅ 182.942µs (SLO: <250.000µs 📉 -26.8%) vs baseline: -1.5% Memory: ✅ 31.733MB (SLO: <34.000MB -6.7%) vs baseline: +5.1% ✅ flush-1000-metricsTime: ✅ 2.221ms (SLO: <2.500ms 📉 -11.1%) vs baseline: -0.9% Memory: ✅ 32.519MB (SLO: <34.500MB -5.7%) vs baseline: +5.1% 🟡 Near SLO Breach (5 suites)🟡 djangosimple - 26/26✅ appsecTime: ✅ 20.497ms (SLO: <22.300ms -8.1%) vs baseline: -0.2% Memory: ✅ 64.448MB (SLO: <66.000MB -2.4%) vs baseline: +4.9% ✅ exception-replay-enabledTime: ✅ 1.343ms (SLO: <1.450ms -7.4%) vs baseline: -0.5% Memory: ✅ 63.439MB (SLO: <66.000MB -3.9%) vs baseline: +4.7% ✅ iastTime: ✅ 20.491ms (SLO: <22.250ms -7.9%) vs baseline: -0.1% Memory: ✅ 64.448MB (SLO: <66.000MB -2.4%) vs baseline: +4.9% ✅ profilerTime: ✅ 15.284ms (SLO: <16.550ms -7.7%) vs baseline: +0.3% Memory: ✅ 52.927MB (SLO: <53.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ span-code-originTime: ✅ 26.245ms (SLO: <28.200ms -6.9%) vs baseline: +0.5% Memory: ✅ 66.596MB (SLO: <68.500MB -2.8%) vs baseline: +4.8% ✅ tracerTime: ✅ 20.471ms (SLO: <21.750ms -5.9%) vs baseline: -0.4% Memory: ✅ 64.468MB (SLO: <66.000MB -2.3%) vs baseline: +4.9% ✅ tracer-and-profilerTime: ✅ 22.140ms (SLO: <23.500ms -5.8%) vs baseline: ~same Memory: ✅ 65.724MB (SLO: <67.000MB 🟡 -1.9%) vs baseline: +4.9% ✅ tracer-dont-create-db-spansTime: ✅ 19.356ms (SLO: <21.500ms -10.0%) vs baseline: -0.2% Memory: ✅ 64.448MB (SLO: <66.000MB -2.4%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 20.481ms (SLO: <21.750ms -5.8%) vs baseline: ~same Memory: ✅ 65.667MB (SLO: <66.000MB 🟡 -0.5%) vs baseline: +4.9% ✅ tracer-no-cachesTime: ✅ 18.406ms (SLO: <19.650ms -6.3%) vs baseline: -0.4% Memory: ✅ 64.427MB (SLO: <66.000MB -2.4%) vs baseline: +4.8% ✅ tracer-no-databasesTime: ✅ 18.743ms (SLO: <20.100ms -6.8%) vs baseline: -0.3% Memory: ✅ 64.409MB (SLO: <66.000MB -2.4%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 20.121ms (SLO: <21.500ms -6.4%) vs baseline: -0.2% Memory: ✅ 64.428MB (SLO: <66.000MB -2.4%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 20.296ms (SLO: <22.000ms -7.7%) vs baseline: ~same Memory: ✅ 64.428MB (SLO: <66.000MB -2.4%) vs baseline: +4.8% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.371ms (SLO: <19.850ms -7.4%) vs baseline: +1.7% Memory: ✅ 64.428MB (SLO: <65.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ errortracking-enabled-userTime: ✅ 18.468ms (SLO: <19.400ms -4.8%) vs baseline: +2.3% Memory: ✅ 64.467MB (SLO: <65.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ tracer-enabledTime: ✅ 18.026ms (SLO: <19.450ms -7.3%) vs baseline: -0.2% Memory: ✅ 64.396MB (SLO: <65.500MB 🟡 -1.7%) vs baseline: +4.7% 🟡 flasksimple - 17/17✅ appsec-getTime: ✅ 4.588ms (SLO: <4.750ms -3.4%) vs baseline: ~same Memory: ✅ 62.305MB (SLO: <64.500MB -3.4%) vs baseline: +4.9% ✅ appsec-postTime: ✅ 6.578ms (SLO: <6.750ms -2.6%) vs baseline: ~same Memory: ✅ 62.246MB (SLO: <64.500MB -3.5%) vs baseline: +4.7% ✅ appsec-telemetryTime: ✅ 4.597ms (SLO: <4.750ms -3.2%) vs baseline: +0.1% Memory: ✅ 62.364MB (SLO: <64.500MB -3.3%) vs baseline: +5.0% ✅ debuggerTime: ✅ 1.861ms (SLO: <2.000ms -7.0%) vs baseline: ~same Memory: ✅ 44.709MB (SLO: <45.000MB 🟡 -0.6%) vs baseline: +4.9% ✅ iast-getTime: ✅ 1.857ms (SLO: <2.000ms -7.2%) vs baseline: -0.2% Memory: ✅ 41.642MB (SLO: <49.000MB 📉 -15.0%) vs baseline: +5.1% ✅ profilerTime: ✅ 1.918ms (SLO: <2.100ms -8.7%) vs baseline: ~same Memory: ✅ 44.296MB (SLO: <46.500MB -4.7%) vs baseline: +4.8% ✅ tracerTime: ✅ 3.374ms (SLO: <3.650ms -7.6%) vs baseline: +0.3% Memory: ✅ 51.413MB (SLO: <53.500MB -3.9%) vs baseline: +5.0% ✅ tracer-nativeTime: ✅ 3.372ms (SLO: <3.650ms -7.6%) vs baseline: -0.1% Memory: ✅ 52.632MB (SLO: <53.500MB 🟡 -1.6%) vs baseline: +4.9% 🟡 flasksqli - 6/6✅ appsec-enabledTime: ✅ 3.951ms (SLO: <4.200ms -5.9%) vs baseline: -0.2% Memory: ✅ 62.628MB (SLO: <66.000MB -5.1%) vs baseline: +4.8% ✅ iast-enabledTime: ✅ 2.447ms (SLO: <2.800ms 📉 -12.6%) vs baseline: -0.8% Memory: ✅ 58.104MB (SLO: <59.000MB 🟡 -1.5%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 2.082ms (SLO: <2.250ms -7.5%) vs baseline: +0.2% Memory: ✅ 51.263MB (SLO: <53.500MB -4.2%) vs baseline: +4.9% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 45.138ms (SLO: <47.150ms -4.3%) vs baseline: -0.4% Memory: ✅ 44.475MB (SLO: <46.500MB -4.4%) vs baseline: +5.0% ✅ add-metricsTime: ✅ 319.493ms (SLO: <344.800ms -7.3%) vs baseline: -0.7% Memory: ✅ 554.029MB (SLO: <562.000MB 🟡 -1.4%) vs baseline: +4.7% ✅ add-tagsTime: ✅ 291.092ms (SLO: <314.000ms -7.3%) vs baseline: -0.2% Memory: ✅ 554.639MB (SLO: <563.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ get-contextTime: ✅ 82.248ms (SLO: <92.350ms 📉 -10.9%) vs baseline: -0.2% Memory: ✅ 39.671MB (SLO: <46.500MB 📉 -14.7%) vs baseline: +4.9% ✅ is-recordingTime: ✅ 43.000ms (SLO: <44.500ms -3.4%) vs baseline: -0.5% Memory: ✅ 43.919MB (SLO: <46.500MB -5.6%) vs baseline: +4.9% ✅ record-exceptionTime: ✅ 61.536ms (SLO: <67.650ms -9.0%) vs baseline: -0.2% Memory: ✅ 39.926MB (SLO: <46.500MB 📉 -14.1%) vs baseline: +4.9% ✅ set-statusTime: ✅ 48.793ms (SLO: <50.400ms -3.2%) vs baseline: -0.4% Memory: ✅ 43.948MB (SLO: <46.500MB -5.5%) vs baseline: +5.0% ✅ startTime: ✅ 42.256ms (SLO: <43.450ms -2.7%) vs baseline: +0.3% Memory: ✅ 43.955MB (SLO: <46.500MB -5.5%) vs baseline: +5.1% ✅ start-finishTime: ✅ 83.431ms (SLO: <88.000ms -5.2%) vs baseline: +0.3% Memory: ✅ 33.856MB (SLO: <46.500MB 📉 -27.2%) vs baseline: +5.0% ✅ start-finish-telemetryTime: ✅ 84.511ms (SLO: <89.000ms -5.0%) vs baseline: -0.2% Memory: ✅ 33.856MB (SLO: <46.500MB 📉 -27.2%) vs baseline: +4.9% ✅ update-nameTime: ✅ 44.680ms (SLO: <45.150ms 🟡 -1.0%) vs baseline: +1.1% Memory: ✅ 44.233MB (SLO: <46.500MB -4.9%) vs baseline: +4.8%
|
When retrieving the list of pending probes, we make the registry return a copy of the collection to avoid mutation during iteration. In the worst case, Python does not warn that the list has been mutated while iterating over it, causing the iteration loop to end prematurely without any sings of errors. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [ ] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 67da5b5)
When retrieving the list of pending probes, we make the registry return a copy of the collection to avoid mutation during iteration. In the worst case, Python does not warn that the list has been mutated while iterating over it, causing the iteration loop to end prematurely without any sings of errors. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [ ] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 67da5b5)
Backport 67da5b5 from #14608 to 3.14. When retrieving the list of pending probes, we make the registry return a copy of the collection to avoid mutation during iteration. In the worst case, Python does not warn that the list has been mutated while iterating over it, causing the iteration loop to end prematurely without any sings of errors. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: Gabriele N. Tornetta <[email protected]>
When retrieving the list of pending probes, we make the registry return a copy of the collection to avoid mutation during iteration. In the worst case, Python does not warn that the list has been mutated while iterating over it, causing the iteration loop to end prematurely without any sings of errors.
Checklist
Reviewer Checklist