-
Notifications
You must be signed in to change notification settings - Fork 469
chore(ci): create an issue and warn on slack when one of the supported branches fails nightly CI #15466
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Juanjo Alvarez <[email protected]>
|
|
Signed-off-by: Juanjo Alvarez <[email protected]>
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 248 ± 3 ms. The average import time from base is: 251 ± 3 ms. The import time difference between this PR and base is: -3.0 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate juanjux/lock-important-branches (6452d69) with baseline main (53c5793) ❌ Test Failures (1 suite)❌ telemetryaddmetric - 29/30✅ 1-count-metric-1-timesTime: ✅ 3.424µs (SLO: <20.000µs 📉 -82.9%) vs baseline: 📈 +15.0% Memory: ✅ 35.036MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.6% ✅ 1-count-metrics-100-timesTime: ✅ 201.298µs (SLO: <220.000µs -8.5%) vs baseline: -1.4% Memory: ✅ 35.055MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.8% ✅ 1-distribution-metric-1-timesTime: ✅ 3.264µs (SLO: <20.000µs 📉 -83.7%) vs baseline: -1.8% Memory: ✅ 35.114MB (SLO: <35.500MB 🟡 -1.1%) vs baseline: +5.8% ✅ 1-distribution-metrics-100-timesTime: ✅ 215.741µs (SLO: <230.000µs -6.2%) vs baseline: -2.4% Memory: ✅ 35.055MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.6% ✅ 1-gauge-metric-1-timesTime: ✅ 2.186µs (SLO: <20.000µs 📉 -89.1%) vs baseline: -1.4% Memory: ✅ 35.055MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.9% ✅ 1-gauge-metrics-100-timesTime: ✅ 137.112µs (SLO: <150.000µs -8.6%) vs baseline: -1.1% Memory: ✅ 35.075MB (SLO: <35.500MB 🟡 -1.2%) vs baseline: +5.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.051µs (SLO: <20.000µs 📉 -84.7%) vs baseline: -1.7% Memory: ✅ 35.154MB (SLO: <35.500MB 🟡 -1.0%) vs baseline: +6.2% ✅ 1-rate-metrics-100-timesTime: ✅ 217.591µs (SLO: <250.000µs 📉 -13.0%) vs baseline: +0.1% Memory: ✅ 35.075MB (SLO: <35.500MB 🟡 -1.2%) vs baseline: +5.6% ✅ 100-count-metrics-100-timesTime: ✅ 20.248ms (SLO: <22.000ms -8.0%) vs baseline: -0.7% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.5% ❌ 100-distribution-metrics-100-timesTime: ❌ 2.313ms (SLO: <2.300ms +0.6%) vs baseline: +1.0% Memory: ✅ 35.114MB (SLO: <35.500MB 🟡 -1.1%) vs baseline: +5.8% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.414ms (SLO: <1.550ms -8.8%) vs baseline: -0.7% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.218ms (SLO: <2.550ms 📉 -13.0%) vs baseline: -0.6% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.4% ✅ flush-1-metricTime: ✅ 4.608µs (SLO: <20.000µs 📉 -77.0%) vs baseline: -1.0% Memory: ✅ 35.173MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +5.3% ✅ flush-100-metricsTime: ✅ 175.151µs (SLO: <250.000µs 📉 -29.9%) vs baseline: +0.5% Memory: ✅ 35.114MB (SLO: <35.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ flush-1000-metricsTime: ✅ 2.177ms (SLO: <2.500ms 📉 -12.9%) vs baseline: -0.3% Memory: ✅ 35.920MB (SLO: <36.500MB 🟡 -1.6%) vs baseline: +5.0% 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.402µs (SLO: <10.000µs 📉 -96.0%) vs baseline: -1.1% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.0% ✅ add_inplace_aspectTime: ✅ 0.407µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +1.5% Memory: ✅ 40.600MB (SLO: <41.500MB -2.2%) vs baseline: +5.8% ✅ add_inplace_noaspectTime: ✅ 0.319µs (SLO: <10.000µs 📉 -96.8%) vs baseline: -1.0% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.7% ✅ add_noaspectTime: ✅ 0.275µs (SLO: <10.000µs 📉 -97.2%) vs baseline: -2.0% Memory: ✅ 40.521MB (SLO: <41.500MB -2.4%) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 1.355µs (SLO: <10.000µs 📉 -86.4%) vs baseline: +0.3% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.3% ✅ bytearray_extend_aspectTime: ✅ 1.506µs (SLO: <10.000µs 📉 -84.9%) vs baseline: -0.6% Memory: ✅ 40.344MB (SLO: <41.500MB -2.8%) vs baseline: +5.5% ✅ bytearray_extend_noaspectTime: ✅ 0.616µs (SLO: <10.000µs 📉 -93.8%) vs baseline: -0.5% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +3.9% ✅ bytearray_noaspectTime: ✅ 0.482µs (SLO: <10.000µs 📉 -95.2%) vs baseline: -0.4% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.7% ✅ bytes_aspectTime: ✅ 1.297µs (SLO: <10.000µs 📉 -87.0%) vs baseline: +1.7% Memory: ✅ 40.364MB (SLO: <41.500MB -2.7%) vs baseline: +4.4% ✅ bytes_noaspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.8% Memory: ✅ 40.501MB (SLO: <41.500MB -2.4%) vs baseline: +5.3% ✅ bytesio_aspectTime: ✅ 1.354µs (SLO: <10.000µs 📉 -86.5%) vs baseline: +1.9% Memory: ✅ 40.128MB (SLO: <41.500MB -3.3%) vs baseline: +4.1% ✅ bytesio_noaspectTime: ✅ 0.498µs (SLO: <10.000µs 📉 -95.0%) vs baseline: -1.3% Memory: ✅ 40.344MB (SLO: <41.500MB -2.8%) vs baseline: +4.6% ✅ capitalize_aspectTime: ✅ 0.735µs (SLO: <10.000µs 📉 -92.6%) vs baseline: -0.5% Memory: ✅ 40.442MB (SLO: <41.500MB -2.5%) vs baseline: +4.5% ✅ capitalize_noaspectTime: ✅ 0.439µs (SLO: <10.000µs 📉 -95.6%) vs baseline: +0.6% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.8% ✅ casefold_aspectTime: ✅ 0.737µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.1% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.4% ✅ casefold_noaspectTime: ✅ 0.364µs (SLO: <10.000µs 📉 -96.4%) vs baseline: -0.9% Memory: ✅ 40.364MB (SLO: <41.500MB -2.7%) vs baseline: +5.2% ✅ decode_aspectTime: ✅ 0.717µs (SLO: <10.000µs 📉 -92.8%) vs baseline: -0.9% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.2% ✅ decode_noaspectTime: ✅ 0.417µs (SLO: <10.000µs 📉 -95.8%) vs baseline: -0.5% Memory: ✅ 40.364MB (SLO: <41.500MB -2.7%) vs baseline: +5.0% ✅ encode_aspectTime: ✅ 0.715µs (SLO: <10.000µs 📉 -92.8%) vs baseline: +1.5% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.0% ✅ encode_noaspectTime: ✅ 0.400µs (SLO: <10.000µs 📉 -96.0%) vs baseline: -1.2% Memory: ✅ 40.580MB (SLO: <41.500MB -2.2%) vs baseline: +5.3% ✅ format_aspectTime: ✅ 3.414µs (SLO: <10.000µs 📉 -65.9%) vs baseline: -1.0% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.9% ✅ format_map_aspectTime: ✅ 3.487µs (SLO: <10.000µs 📉 -65.1%) vs baseline: -2.6% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +4.5% ✅ format_map_noaspectTime: ✅ 0.779µs (SLO: <10.000µs 📉 -92.2%) vs baseline: +0.3% Memory: ✅ 40.265MB (SLO: <41.500MB -3.0%) vs baseline: +4.5% ✅ format_noaspectTime: ✅ 0.592µs (SLO: <10.000µs 📉 -94.1%) vs baseline: -1.3% Memory: ✅ 40.344MB (SLO: <41.500MB -2.8%) vs baseline: +4.8% ✅ index_aspectTime: ✅ 0.358µs (SLO: <10.000µs 📉 -96.4%) vs baseline: +1.0% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.5% ✅ index_noaspectTime: ✅ 0.279µs (SLO: <10.000µs 📉 -97.2%) vs baseline: +1.8% Memory: ✅ 40.541MB (SLO: <41.500MB -2.3%) vs baseline: +5.5% ✅ join_aspectTime: ✅ 1.345µs (SLO: <10.000µs 📉 -86.6%) vs baseline: +0.4% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.3% ✅ join_noaspectTime: ✅ 0.495µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.4% Memory: ✅ 40.541MB (SLO: <41.500MB -2.3%) vs baseline: +4.8% ✅ ljust_aspectTime: ✅ 2.928µs (SLO: <20.000µs 📉 -85.4%) vs baseline: 📈 +14.6% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.8% ✅ ljust_noaspectTime: ✅ 0.403µs (SLO: <10.000µs 📉 -96.0%) vs baseline: -2.2% Memory: ✅ 40.403MB (SLO: <41.500MB -2.6%) vs baseline: +5.0% ✅ lower_aspectTime: ✅ 2.328µs (SLO: <10.000µs 📉 -76.7%) vs baseline: +3.9% Memory: ✅ 40.108MB (SLO: <41.500MB -3.4%) vs baseline: +4.5% ✅ lower_noaspectTime: ✅ 0.363µs (SLO: <10.000µs 📉 -96.4%) vs baseline: -1.8% Memory: ✅ 40.521MB (SLO: <41.500MB -2.4%) vs baseline: +5.5% ✅ lstrip_aspectTime: ✅ 2.261µs (SLO: <20.000µs 📉 -88.7%) vs baseline: +1.2% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.7% ✅ lstrip_noaspectTime: ✅ 0.382µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.6% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.7% ✅ modulo_aspectTime: ✅ 1.049µs (SLO: <10.000µs 📉 -89.5%) vs baseline: +4.3% Memory: ✅ 40.108MB (SLO: <41.500MB -3.4%) vs baseline: +3.5% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.548µs (SLO: <10.000µs 📉 -84.5%) vs baseline: -0.2% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 0.978µs (SLO: <10.000µs 📉 -90.2%) vs baseline: -0.3% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.8% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.257µs (SLO: <10.000µs 📉 -87.4%) vs baseline: +2.6% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.8% ✅ modulo_noaspectTime: ✅ 0.628µs (SLO: <10.000µs 📉 -93.7%) vs baseline: +0.2% Memory: ✅ 40.482MB (SLO: <41.500MB -2.5%) vs baseline: +5.5% ✅ replace_aspectTime: ✅ 4.954µs (SLO: <10.000µs 📉 -50.5%) vs baseline: +0.7% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +3.9% ✅ replace_noaspectTime: ✅ 0.457µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -0.9% Memory: ✅ 40.462MB (SLO: <41.500MB -2.5%) vs baseline: +5.2% ✅ repr_aspectTime: ✅ 0.907µs (SLO: <10.000µs 📉 -90.9%) vs baseline: ~same Memory: ✅ 40.147MB (SLO: <41.500MB -3.3%) vs baseline: +4.0% ✅ repr_noaspectTime: ✅ 0.413µs (SLO: <10.000µs 📉 -95.9%) vs baseline: -1.6% Memory: ✅ 40.482MB (SLO: <41.500MB -2.5%) vs baseline: +4.8% ✅ rstrip_aspectTime: ✅ 1.935µs (SLO: <20.000µs 📉 -90.3%) vs baseline: +0.6% Memory: ✅ 40.166MB (SLO: <41.500MB -3.2%) vs baseline: +4.2% ✅ rstrip_noaspectTime: ✅ 0.377µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -1.3% Memory: ✅ 40.344MB (SLO: <41.500MB -2.8%) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -1.1% Memory: ✅ 40.128MB (SLO: <41.500MB -3.3%) vs baseline: +4.3% ✅ slice_noaspectTime: ✅ 0.444µs (SLO: <10.000µs 📉 -95.6%) vs baseline: -0.7% Memory: ✅ 40.403MB (SLO: <41.500MB -2.6%) vs baseline: +5.4% ✅ stringio_aspectTime: ✅ 1.539µs (SLO: <10.000µs 📉 -84.6%) vs baseline: +0.7% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.0% ✅ stringio_noaspectTime: ✅ 0.719µs (SLO: <10.000µs 📉 -92.8%) vs baseline: -0.1% Memory: ✅ 40.383MB (SLO: <41.500MB -2.7%) vs baseline: +5.4% ✅ strip_aspectTime: ✅ 2.245µs (SLO: <20.000µs 📉 -88.8%) vs baseline: +0.7% Memory: ✅ 40.364MB (SLO: <41.500MB -2.7%) vs baseline: +5.2% ✅ strip_noaspectTime: ✅ 0.383µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.6% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +4.8% ✅ swapcase_aspectTime: ✅ 2.775µs (SLO: <10.000µs 📉 -72.3%) vs baseline: 📈 +13.5% Memory: ✅ 40.128MB (SLO: <41.500MB -3.3%) vs baseline: +3.8% ✅ swapcase_noaspectTime: ✅ 0.540µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.2% Memory: ✅ 40.382MB (SLO: <41.500MB -2.7%) vs baseline: +5.1% ✅ title_aspectTime: ✅ 2.447µs (SLO: <10.000µs 📉 -75.5%) vs baseline: +3.3% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.9% ✅ title_noaspectTime: ✅ 0.501µs (SLO: <10.000µs 📉 -95.0%) vs baseline: ~same Memory: ✅ 40.442MB (SLO: <41.500MB -2.5%) vs baseline: +5.1% ✅ translate_aspectTime: ✅ 3.307µs (SLO: <10.000µs 📉 -66.9%) vs baseline: -0.3% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.0% ✅ translate_noaspectTime: ✅ 1.038µs (SLO: <10.000µs 📉 -89.6%) vs baseline: -0.3% Memory: ✅ 40.029MB (SLO: <41.500MB -3.5%) vs baseline: +4.1% ✅ upper_aspectTime: ✅ 2.301µs (SLO: <10.000µs 📉 -77.0%) vs baseline: +1.8% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +5.2% ✅ upper_noaspectTime: ✅ 0.367µs (SLO: <10.000µs 📉 -96.3%) vs baseline: ~same Memory: ✅ 40.462MB (SLO: <41.500MB -2.5%) vs baseline: +5.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.159µs (SLO: <10.000µs 📉 -48.4%) vs baseline: 📈 +25.2% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ ospathbasename_noaspectTime: ✅ 1.095µs (SLO: <10.000µs 📉 -89.1%) vs baseline: +0.6% Memory: ✅ 40.305MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +5.3% ✅ ospathjoin_aspectTime: ✅ 6.154µs (SLO: <10.000µs 📉 -38.5%) vs baseline: -0.9% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +5.1% ✅ ospathjoin_noaspectTime: ✅ 2.281µs (SLO: <10.000µs 📉 -77.2%) vs baseline: -0.8% Memory: ✅ 40.265MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +5.0% ✅ ospathnormcase_aspectTime: ✅ 3.423µs (SLO: <10.000µs 📉 -65.8%) vs baseline: -0.3% Memory: ✅ 40.128MB (SLO: <41.000MB -2.1%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 0.568µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -1.1% Memory: ✅ 40.265MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +5.2% ✅ ospathsplit_aspectTime: ✅ 4.732µs (SLO: <10.000µs 📉 -52.7%) vs baseline: -1.0% Memory: ✅ 40.167MB (SLO: <41.000MB -2.0%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 1.600µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +0.5% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.9% ✅ ospathsplitdrive_aspectTime: ✅ 3.584µs (SLO: <10.000µs 📉 -64.2%) vs baseline: -1.2% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ ospathsplitdrive_noaspectTime: ✅ 0.695µs (SLO: <10.000µs 📉 -93.0%) vs baseline: -0.8% Memory: ✅ 40.285MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 4.503µs (SLO: <10.000µs 📉 -55.0%) vs baseline: +0.2% Memory: ✅ 40.206MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 1.371µs (SLO: <10.000µs 📉 -86.3%) vs baseline: -0.8% Memory: ✅ 40.167MB (SLO: <41.000MB -2.0%) vs baseline: +4.7% 🟡 Near SLO Breach (16 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
Signed-off-by: Juanjo Alvarez <[email protected]>
Signed-off-by: Juanjo Alvarez <[email protected]>
Signed-off-by: Juanjo Alvarez <[email protected]>
|
We have a monitor configured to do this instead. |
Description
Note: Both the github issue creation and the slack channel warning will be disabled until we merge and check that the test-run automation is working to avoid spamming the channel/github if not. Both would be enabled on a further PR.