Skip to content

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Jun 19, 2024

It looks like #58867 caused some regressions; vscode benchmark crashes on --diagnostics, and even if that crash is fixed, it fails to resolve lib and gets more errors.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 19, 2024
@jakebailey
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 19, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user tests with tsc comparing main and refs/pull/58935/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,673k (± 0.76%) 192,749k (± 0.79%) ~ 192,112k 195,843k p=0.575 n=6
Parse Time 1.94s (± 2.19%) 1.95s (± 1.47%) ~ 1.90s 1.98s p=0.745 n=6
Bind Time 1.06s (± 0.77%) 1.05s (± 1.15%) ~ 1.04s 1.07s p=0.157 n=6
Check Time 13.84s (± 0.54%) 13.83s (± 0.43%) ~ 13.72s 13.88s p=0.466 n=6
Emit Time 4.10s (± 3.50%) 4.03s (± 1.20%) ~ 3.94s 4.07s p=0.744 n=6
Total Time 20.94s (± 0.76%) 20.87s (± 0.34%) ~ 20.77s 20.94s p=0.521 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,298k (± 0.00%) 1,218,300k (± 0.00%) ~ 1,218,249k 1,218,348k p=1.000 n=6
Parse Time 7.95s (± 0.33%) 7.94s (± 0.57%) ~ 7.89s 8.00s p=0.573 n=6
Bind Time 2.23s (± 0.87%) 2.23s (± 0.44%) ~ 2.22s 2.25s p=0.934 n=6
Check Time 35.74s (± 0.33%) 35.68s (± 0.36%) ~ 35.50s 35.80s p=0.377 n=6
Emit Time 16.11s (± 0.51%) 16.11s (± 0.35%) ~ 16.03s 16.19s p=1.000 n=6
Total Time 62.03s (± 0.22%) 61.96s (± 0.21%) ~ 61.77s 62.14s p=0.423 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,420 2,133,420 ~ ~ ~ p=1.000 n=6
Types 926,456 926,456 ~ ~ ~ p=1.000 n=6
Memory used 2,115,588k (± 0.00%) 2,115,476k (± 0.00%) -112k (- 0.01%) 2,115,389k 2,115,656k p=0.031 n=6
Parse Time 7.85s (± 0.39%) 7.85s (± 0.56%) ~ 7.80s 7.90s p=0.808 n=6
Bind Time 2.74s (± 0.50%) 2.74s (± 0.64%) ~ 2.72s 2.76s p=1.000 n=6
Check Time 83.52s (± 0.40%) 83.32s (± 0.45%) ~ 82.88s 83.84s p=0.378 n=6
Emit Time 0.15s (± 3.53%) 0.16s (± 4.65%) ~ 0.15s 0.17s p=0.137 n=6
Total Time 94.26s (± 0.35%) 94.06s (± 0.37%) ~ 93.65s 94.53s p=0.378 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,762 1,230,692 -70 (- 0.01%) ~ ~ p=0.001 n=6
Types 261,187 261,165 -22 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,345,950k (± 0.05%) 2,356,136k (± 1.04%) ~ 2,344,282k 2,406,139k p=0.378 n=6
Parse Time 6.02s (± 0.86%) 5.95s (± 0.89%) ~ 5.90s 6.04s p=0.054 n=6
Bind Time 2.26s (± 1.23%) 2.27s (± 1.50%) ~ 2.23s 2.32s p=1.000 n=6
Check Time 39.66s (± 0.31%) 39.75s (± 0.20%) ~ 39.63s 39.86s p=0.128 n=6
Emit Time 3.18s (± 2.57%) 3.13s (± 3.58%) ~ 3.01s 3.32s p=0.229 n=6
Total Time 51.14s (± 0.34%) 51.13s (± 0.22%) ~ 50.94s 51.28s p=0.688 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,762 1,230,692 -70 (- 0.01%) ~ ~ p=0.001 n=6
Types 261,187 261,165 -22 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 2,432,208k (± 1.00%) 2,423,048k (± 0.02%) ~ 2,422,320k 2,423,641k p=0.689 n=6
Parse Time 7.73s (± 0.84%) 7.65s (± 0.58%) ~ 7.58s 7.70s p=0.066 n=6
Bind Time 2.50s (± 2.04%) 2.47s (± 1.04%) ~ 2.43s 2.50s p=0.374 n=6
Check Time 49.44s (± 0.47%) 49.42s (± 0.31%) ~ 49.15s 49.59s p=0.936 n=6
Emit Time 3.88s (± 2.87%) 3.83s (± 1.02%) ~ 3.78s 3.90s p=0.470 n=6
Total Time 63.57s (± 0.52%) 63.39s (± 0.25%) ~ 63.14s 63.57s p=0.298 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,573 258,573 ~ ~ ~ p=1.000 n=6
Types 104,819 104,819 ~ ~ ~ p=1.000 n=6
Memory used 428,184k (± 0.02%) 428,194k (± 0.00%) ~ 428,173k 428,215k p=0.230 n=6
Parse Time 3.31s (± 0.90%) 3.34s (± 0.92%) ~ 3.30s 3.39s p=0.196 n=6
Bind Time 1.30s (± 0.90%) 1.33s (± 1.23%) +0.03s (+ 1.92%) 1.31s 1.35s p=0.023 n=6
Check Time 17.80s (± 0.35%) 17.77s (± 0.23%) ~ 17.73s 17.83s p=0.229 n=6
Emit Time 1.35s (± 0.86%) 1.36s (± 0.72%) ~ 1.35s 1.37s p=0.181 n=6
Total Time 23.77s (± 0.32%) 23.79s (± 0.20%) ~ 23.72s 23.84s p=0.572 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,454k (± 0.04%) 369,508k (± 0.04%) ~ 369,383k 369,764k p=0.575 n=6
Parse Time 2.75s (± 0.95%) 2.76s (± 1.84%) ~ 2.70s 2.84s p=1.000 n=6
Bind Time 1.58s (± 0.62%) 1.58s (± 0.69%) ~ 1.57s 1.60s p=0.865 n=6
Check Time 15.42s (± 0.44%) 15.40s (± 0.32%) ~ 15.32s 15.45s p=0.747 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.75s (± 0.36%) 19.74s (± 0.44%) ~ 19.61s 19.84s p=1.000 n=6
vscode - node (v18.15.0, x64)
Errors 0 139 ~ ~ ~ p=1.000 n=0+6
Symbols 0 2,846,102 ~ ~ ~ p=1.000 n=0+6
Types 0 973,724 ~ ~ ~ p=1.000 n=0+6
Memory used 0k 3,038,830k (± 0.00%) ~ 3,038,741k 3,038,922k p=1.000 n=0+6
Parse Time 0s 16.71s (± 0.45%) ~ 16.65s 16.86s p=1.000 n=0+6
Bind Time 0s 5.10s (± 0.57%) ~ 5.06s 5.14s p=1.000 n=0+6
Check Time 0s 90.86s (± 1.91%) ~ 89.87s 94.39s p=1.000 n=0+6
Emit Time 0s 28.41s (± 6.54%) ~ 24.62s 29.26s p=1.000 n=0+6
Total Time 0s 141.08s (± 0.16%) ~ 140.74s 141.41s p=1.000 n=0+6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,116 267,116 ~ ~ ~ p=1.000 n=6
Types 108,756 108,756 ~ ~ ~ p=1.000 n=6
Memory used 411,596k (± 0.01%) 411,638k (± 0.03%) ~ 411,474k 411,812k p=0.575 n=6
Parse Time 4.70s (± 0.54%) 4.70s (± 0.51%) ~ 4.67s 4.74s p=0.808 n=6
Bind Time 2.08s (± 0.95%) 2.08s (± 0.88%) ~ 2.06s 2.11s p=0.739 n=6
Check Time 20.75s (± 0.51%) 20.86s (± 0.39%) ~ 20.75s 20.95s p=0.093 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.54s (± 0.38%) 27.64s (± 0.31%) ~ 27.52s 27.73s p=0.127 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,898k (± 0.10%) 462,146k (± 0.02%) ~ 462,035k 462,309k p=0.810 n=6
Parse Time 2.64s (± 0.39%) 2.63s (± 0.62%) ~ 2.61s 2.65s p=0.869 n=6
Bind Time 0.99s (± 0.52%) 0.98s -0.01s (- 0.68%) ~ ~ p=0.025 n=6
Check Time 15.18s (± 0.36%) 15.24s (± 0.43%) ~ 15.14s 15.33s p=0.228 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.81s (± 0.30%) 18.85s (± 0.39%) ~ 18.74s 18.94s p=0.261 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58935/merge:

Everything looks good!

@jakebailey jakebailey marked this pull request as ready for review June 19, 2024 21:30
@jakebailey jakebailey changed the title Revert "Fix incorrect condition of noLib (#58867)" Revert "Fix incorrect condition of noLib" (PR 58867) Jun 19, 2024
@jakebailey
Copy link
Member Author

Going to merge this to unblock benchmarking (per DM); I have a change locally that would also fix the original crash the PR solved, but I think there's more going on here.

@jakebailey jakebailey merged commit aa249c0 into microsoft:main Jun 19, 2024
@jakebailey jakebailey deleted the revert-58867 branch June 19, 2024 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants