Skip to content

Shortcut relations in getNarrowedTypeWorker when comparing literals #56072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Shortcut relations in getNarrowedTypeWorker when comparing literals #56072

wants to merge 7 commits into from

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Oct 11, 2023 •
edited
Loading

For #55948

When narrowing large unions of literals, we do four different relation calls which appear to be quite costly. By checking ahead of time that we're going to relate two literals, we can skip this work by comparing the values directly and only once (as the result shouldn't be order dependent).

This leads to a significant speedup on #55948:

Benchmark 1: main
  Time (mean ± σ):     12.456 s ±  0.851 s    [User: 12.929 s, System: 0.059 s]
  Range (min … max):   11.219 s … 13.428 s    10 runs
 
Benchmark 2: PR
  Time (mean ± σ):      2.973 s ±  0.053 s    [User: 3.434 s, System: 0.057 s]
  Range (min … max):    2.906 s …  3.073 s    10 runs
 
Summary
  'PR' ran
    4.19 ± 0.30 times faster than 'main'

This PR passes all of the tests, but I think my implementation is probably wrong for some edge cases. Like, I don't validate that the literal types are the same between the two, just their values, which is wrong. But, doesn't break? Fixed that edge case.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 11, 2023
@jakebailey
Copy link
Member Author

@typescript-bot test top100
@typescript-bot user test this
@typescript-bot run dt
@typescript-bot perf test this
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at 54d0bc7. You can monitor the build here.

@jakebailey jakebailey mentioned this pull request Oct 11, 2023
Open
@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 11, 2023 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/158179/artifacts?artifactName=tgz&fileId=130E9BE142C9389724046D51A9B44F05AB5048F38496A635A91CD40D59518E7802&fileName=/typescript-5.3.0-insiders.20231011.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/56072/merge:

There were infrastructure failures potentially unrelated to your change:

  • 3 instances of "Package install failed"
  • 1 instance of "Unknown failure"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,117k (± 0.01%) 295,130k (± 0.01%) ~ 295,096k 295,184k p=0.688 n=6
Parse Time 2.63s (± 0.57%) 2.64s (± 0.31%) ~ 2.63s 2.65s p=0.236 n=6
Bind Time 0.84s (± 1.17%) 0.84s (± 0.97%) ~ 0.83s 0.85s p=0.862 n=6
Check Time 8.07s (± 0.11%) 8.08s (± 0.31%) ~ 8.04s 8.11s p=0.459 n=6
Emit Time 7.05s (± 0.11%) 7.03s (± 0.42%) ~ 6.99s 7.06s p=0.406 n=6
Total Time 18.59s (± 0.10%) 18.59s (± 0.26%) ~ 18.52s 18.65s p=0.747 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,611k (± 1.59%) 194,495k (± 1.53%) ~ 190,608k 196,470k p=0.810 n=6
Parse Time 1.34s (± 0.47%) 1.34s (± 0.56%) ~ 1.33s 1.35s p=0.718 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.56%) ~ 0.72s 0.73s p=0.405 n=6
Check Time 9.13s (± 0.26%) 9.10s (± 0.44%) ~ 9.06s 9.15s p=0.257 n=6
Emit Time 2.63s (± 0.20%) 2.63s (± 0.44%) ~ 2.61s 2.64s p=0.195 n=6
Total Time 13.83s (± 0.10%) 13.80s (± 0.30%) ~ 13.75s 13.85s p=0.512 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,305k (± 0.00%) 347,317k (± 0.00%) ~ 347,304k 347,321k p=0.054 n=6
Parse Time 2.46s (± 0.31%) 2.46s (± 0.33%) ~ 2.45s 2.47s p=0.306 n=6
Bind Time 0.94s (± 0.43%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=0.405 n=6
Check Time 6.91s (± 0.25%) 6.92s (± 0.44%) ~ 6.87s 6.96s p=0.568 n=6
Emit Time 4.01s (± 0.34%) 4.03s (± 0.53%) ~ 4.01s 4.07s p=0.391 n=6
Total Time 14.33s (± 0.21%) 14.36s (± 0.33%) ~ 14.32s 14.45s p=0.627 n=6
TFS - node (v18.15.0, x64)
Memory used 302,546k (± 0.01%) 302,548k (± 0.01%) ~ 302,525k 302,576k p=0.936 n=6
Parse Time 1.99s (± 0.45%) 1.99s (± 1.00%) ~ 1.97s 2.02s p=0.934 n=6
Bind Time 1.01s (± 0.97%) 1.00s (± 0.41%) ~ 1.00s 1.01s p=0.213 n=6
Check Time 6.25s (± 0.22%) 6.27s (± 0.41%) ~ 6.23s 6.29s p=0.462 n=6
Emit Time 3.56s (± 0.98%) 3.56s (± 0.48%) ~ 3.54s 3.59s p=1.000 n=6
Total Time 12.82s (± 0.33%) 12.82s (± 0.23%) ~ 12.77s 12.85s p=1.000 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,507k (± 0.00%) 470,488k (± 0.00%) ~ 470,457k 470,512k p=0.066 n=6
Parse Time 2.57s (± 0.75%) 2.58s (± 0.45%) ~ 2.57s 2.60s p=0.410 n=6
Bind Time 1.00s (± 1.33%) 1.00s (± 1.60%) ~ 0.97s 1.02s p=0.720 n=6
Check Time 16.57s (± 0.19%) 16.58s (± 0.43%) ~ 16.52s 16.72s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.14s (± 0.23%) 20.16s (± 0.35%) ~ 20.10s 20.29s p=0.687 n=6
xstate - node (v18.15.0, x64)
Memory used 512,620k (± 0.01%) 512,637k (± 0.00%) ~ 512,615k 512,674k p=0.173 n=6
Parse Time 3.27s (± 0.42%) 3.27s (± 0.25%) ~ 3.26s 3.28s p=0.933 n=6
Bind Time 1.55s (± 0.48%) 1.55s (± 0.41%) ~ 1.54s 1.56s p=0.718 n=6
Check Time 2.86s (± 0.84%) 2.86s (± 0.49%) ~ 2.84s 2.88s p=0.935 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=1.000 n=6
Total Time 7.76s (± 0.25%) 7.76s (± 0.26%) ~ 7.73s 7.79s p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,354ms (± 0.94%) 2,341ms (± 0.21%) ~ 2,336ms 2,350ms p=0.575 n=6
Req 2 - geterr 5,393ms (± 1.66%) 5,404ms (± 1.51%) ~ 5,285ms 5,475ms p=0.575 n=6
Req 3 - references 328ms (± 0.86%) 326ms (± 0.36%) ~ 325ms 328ms p=0.558 n=6
Req 4 - navto 276ms (± 1.14%) 277ms (± 1.51%) ~ 273ms 282ms p=0.805 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 85ms (± 8.94%) 85ms (± 8.98%) ~ 75ms 90ms p=0.491 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,490ms (± 0.67%) 2,452ms (± 1.41%) ~ 2,410ms 2,505ms p=0.054 n=6
Req 2 - geterr 4,086ms (± 1.47%) 4,185ms (± 1.44%) ~ 4,063ms 4,225ms p=0.066 n=6
Req 3 - references 341ms (± 1.21%) 335ms (± 1.06%) -7ms (- 1.90%) 333ms 342ms p=0.039 n=6
Req 4 - navto 285ms (± 0.37%) 283ms (± 0.62%) -2ms (- 0.70%) 279ms 284ms p=0.024 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 85ms (± 5.32%) 77ms (± 6.31%) 🟩-8ms (- 9.39%) 74ms 87ms p=0.032 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,597ms (± 0.38%) 2,595ms (± 0.62%) ~ 2,562ms 2,604ms p=0.521 n=6
Req 2 - geterr 1,700ms (± 3.31%) 1,682ms (± 2.45%) ~ 1,610ms 1,728ms p=0.936 n=6
Req 3 - references 111ms (± 7.25%) 112ms (± 7.96%) ~ 105ms 126ms p=1.000 n=6
Req 4 - navto 361ms (± 1.29%) 359ms (± 0.33%) ~ 358ms 361ms p=0.803 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 305ms (± 1.85%) 306ms (± 1.13%) ~ 300ms 310ms p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.38ms (± 0.18%) 152.26ms (± 0.18%) -0.13ms (- 0.08%) 151.26ms 155.17ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.82ms (± 0.16%) 227.77ms (± 0.17%) -0.05ms (- 0.02%) 226.26ms 233.24ms p=0.046 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.64ms (± 0.17%) 228.59ms (± 0.19%) ~ 226.96ms 240.71ms p=0.169 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 228.89ms (± 0.17%) 228.94ms (± 0.16%) ~ 227.44ms 233.23ms p=0.229 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/56072/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 12, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 629de0d. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,131k (± 0.01%) 295,104k (± 0.02%) ~ 294,993k 295,136k p=0.575 n=6
Parse Time 2.63s (± 0.21%) 2.64s (± 0.61%) ~ 2.61s 2.65s p=0.357 n=6
Bind Time 0.83s (± 0.62%) 0.84s (± 0.97%) ~ 0.83s 0.85s p=0.523 n=6
Check Time 8.06s (± 0.36%) 8.06s (± 0.31%) ~ 8.02s 8.09s p=0.683 n=6
Emit Time 7.03s (± 0.49%) 7.05s (± 0.17%) ~ 7.03s 7.06s p=0.466 n=6
Total Time 18.57s (± 0.23%) 18.58s (± 0.13%) ~ 18.56s 18.63s p=0.418 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 191,611k (± 1.24%) 190,628k (± 0.01%) ~ 190,603k 190,656k p=0.229 n=6
Parse Time 1.35s (± 1.11%) 1.35s (± 0.90%) ~ 1.34s 1.37s p=0.343 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.13s (± 0.28%) 9.13s (± 0.54%) ~ 9.06s 9.21s p=0.624 n=6
Emit Time 2.63s (± 0.44%) 2.63s (± 0.61%) ~ 2.62s 2.66s p=0.438 n=6
Total Time 13.85s (± 0.11%) 13.83s (± 0.30%) ~ 13.77s 13.90s p=0.226 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,316k (± 0.01%) 347,324k (± 0.01%) ~ 347,289k 347,362k p=0.575 n=6
Parse Time 2.45s (± 0.21%) 2.46s (± 0.54%) ~ 2.44s 2.47s p=0.546 n=6
Bind Time 0.94s (± 0.43%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=0.405 n=6
Check Time 6.92s (± 0.46%) 6.93s (± 0.73%) ~ 6.88s 7.00s p=1.000 n=6
Emit Time 4.02s (± 0.41%) 4.00s (± 0.26%) -0.02s (- 0.50%) 3.99s 4.02s p=0.049 n=6
Total Time 14.33s (± 0.26%) 14.33s (± 0.38%) ~ 14.26s 14.41s p=0.748 n=6
TFS - node (v18.15.0, x64)
Memory used 302,568k (± 0.01%) 302,538k (± 0.00%) -31k (- 0.01%) 302,524k 302,550k p=0.013 n=6
Parse Time 2.01s (± 1.03%) 2.00s (± 0.94%) ~ 1.96s 2.01s p=0.514 n=6
Bind Time 1.01s (± 0.97%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=0.102 n=6
Check Time 6.25s (± 0.29%) 6.25s (± 0.35%) ~ 6.22s 6.28s p=0.685 n=6
Emit Time 3.56s (± 0.42%) 3.55s (± 0.53%) ~ 3.53s 3.58s p=0.934 n=6
Total Time 12.82s (± 0.20%) 12.80s (± 0.29%) ~ 12.75s 12.85s p=0.470 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,499k (± 0.01%) 470,489k (± 0.00%) ~ 470,465k 470,517k p=0.630 n=6
Parse Time 2.59s (± 1.02%) 2.57s (± 0.62%) ~ 2.55s 2.59s p=0.120 n=6
Bind Time 0.99s (± 0.64%) 0.99s (± 0.84%) ~ 0.99s 1.01s p=0.340 n=6
Check Time 16.61s (± 0.55%) 16.58s (± 0.13%) ~ 16.56s 16.62s p=0.258 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.19s (± 0.54%) 20.15s (± 0.18%) ~ 20.12s 20.21s p=0.199 n=6
xstate - node (v18.15.0, x64)
Memory used 512,615k (± 0.01%) 512,658k (± 0.01%) ~ 512,617k 512,717k p=0.128 n=6
Parse Time 3.26s (± 0.27%) 3.27s (± 0.30%) ~ 3.26s 3.28s p=0.082 n=6
Bind Time 1.55s (± 0.53%) 1.54s (± 0.35%) ~ 1.54s 1.55s p=0.088 n=6
Check Time 2.88s (± 0.90%) 2.88s (± 0.87%) ~ 2.86s 2.93s p=0.687 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=1.000 n=6
Total Time 7.77s (± 0.41%) 7.78s (± 0.42%) ~ 7.75s 7.84s p=0.327 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,381ms (± 0.94%) 2,366ms (± 1.62%) ~ 2,318ms 2,407ms p=0.810 n=6
Req 2 - geterr 5,316ms (± 1.45%) 5,370ms (± 1.58%) ~ 5,287ms 5,453ms p=0.229 n=6
Req 3 - references 331ms (± 1.59%) 327ms (± 0.95%) ~ 324ms 333ms p=0.373 n=6
Req 4 - navto 280ms (± 1.42%) 277ms (± 1.36%) ~ 273ms 282ms p=0.224 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 79ms (± 7.24%) 83ms (± 9.69%) ~ 75ms 91ms p=0.462 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,492ms (± 0.53%) 2,480ms (± 0.93%) ~ 2,440ms 2,500ms p=0.423 n=6
Req 2 - geterr 4,111ms (± 1.79%) 4,107ms (± 1.78%) ~ 4,052ms 4,204ms p=0.471 n=6
Req 3 - references 340ms (± 1.46%) 340ms (± 1.61%) ~ 332ms 344ms p=0.870 n=6
Req 4 - navto 284ms (± 0.44%) 284ms (± 0.36%) ~ 282ms 285ms p=0.560 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 83ms (± 7.02%) 83ms (± 6.52%) ~ 76ms 87ms p=1.000 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,588ms (± 0.69%) 2,599ms (± 0.41%) ~ 2,587ms 2,611ms p=0.422 n=6
Req 2 - geterr 1,692ms (± 1.81%) 1,715ms (± 2.38%) ~ 1,657ms 1,763ms p=0.423 n=6
Req 3 - references 106ms (± 0.52%) 118ms (± 7.85%) 🔻+12ms (+11.53%) 106ms 127ms p=0.015 n=6
Req 4 - navto 359ms (± 0.25%) 360ms (± 1.09%) ~ 357ms 368ms p=0.804 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 305ms (± 2.33%) 308ms (± 2.88%) ~ 296ms 319ms p=0.748 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.49ms (± 0.18%) 152.27ms (± 0.17%) -0.22ms (- 0.15%) 151.27ms 155.68ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.89ms (± 0.14%) 227.89ms (± 0.17%) ~ 226.49ms 232.25ms p=0.251 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.93ms (± 0.15%) 229.02ms (± 0.20%) ~ 227.43ms 240.23ms p=0.185 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 228.19ms (± 0.15%) 228.21ms (± 0.16%) ~ 226.82ms 232.43ms p=0.680 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

weswigham
weswigham reviewed Oct 12, 2023
View reviewed changes
@jakebailey
Copy link
Member Author

@typescript-bot test top200
@typescript-bot user test this
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 12, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 12, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 12, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/56072/merge:

There were infrastructure failures potentially unrelated to your change:

  • 3 instances of "Package install failed"
  • 1 instance of "Unknown failure"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,149k (± 0.01%) 295,134k (± 0.02%) ~ 295,082k 295,192k p=1.000 n=6
Parse Time 2.65s (± 0.32%) 2.63s (± 0.71%) ~ 2.60s 2.65s p=0.065 n=6
Bind Time 0.83s (± 0.66%) 0.84s (± 1.30%) ~ 0.83s 0.85s p=0.487 n=6
Check Time 8.05s (± 0.33%) 8.08s (± 0.34%) +0.03s (+ 0.41%) 8.06s 8.12s p=0.035 n=6
Emit Time 7.07s (± 0.33%) 7.06s (± 0.39%) ~ 7.03s 7.10s p=0.683 n=6
Total Time 18.60s (± 0.16%) 18.61s (± 0.20%) ~ 18.57s 18.66s p=0.625 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,550k (± 1.54%) 191,599k (± 1.23%) ~ 190,592k 196,433k p=0.872 n=6
Parse Time 1.34s (± 0.38%) 1.36s (± 1.27%) ~ 1.34s 1.38s p=0.142 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.14s (± 0.37%) 9.20s (± 0.52%) +0.06s (+ 0.66%) 9.13s 9.26s p=0.045 n=6
Emit Time 2.63s (± 0.39%) 2.63s (± 0.46%) ~ 2.61s 2.64s p=0.735 n=6
Total Time 13.85s (± 0.35%) 13.93s (± 0.34%) +0.08s (+ 0.58%) 13.84s 13.97s p=0.036 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,312k (± 0.01%) 347,345k (± 0.00%) +33k (+ 0.01%) 347,338k 347,358k p=0.030 n=6
Parse Time 2.45s (± 0.87%) 2.45s (± 0.61%) ~ 2.44s 2.47s p=0.869 n=6
Bind Time 0.95s (± 1.68%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=0.074 n=6
Check Time 6.95s (± 0.46%) 6.96s (± 0.82%) ~ 6.90s 7.03s p=0.936 n=6
Emit Time 4.03s (± 0.49%) 4.01s (± 0.37%) ~ 4.00s 4.04s p=0.251 n=6
Total Time 14.37s (± 0.34%) 14.37s (± 0.27%) ~ 14.32s 14.42s p=0.685 n=6
TFS - node (v18.15.0, x64)
Memory used 302,546k (± 0.01%) 302,591k (± 0.00%) +45k (+ 0.01%) 302,567k 302,601k p=0.008 n=6
Parse Time 2.00s (± 0.60%) 1.99s (± 0.88%) ~ 1.98s 2.02s p=0.251 n=6
Bind Time 1.00s (± 0.81%) 1.01s (± 0.54%) +0.01s (+ 1.16%) 1.01s 1.02s p=0.027 n=6
Check Time 6.27s (± 0.23%) 6.27s (± 0.35%) ~ 6.23s 6.29s p=0.869 n=6
Emit Time 3.56s (± 0.38%) 3.57s (± 0.46%) ~ 3.55s 3.59s p=0.088 n=6
Total Time 12.83s (± 0.08%) 12.85s (± 0.25%) ~ 12.81s 12.90s p=0.744 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,487k (± 0.00%) 470,494k (± 0.00%) ~ 470,467k 470,512k p=0.470 n=6
Parse Time 2.57s (± 0.29%) 2.57s (± 0.40%) ~ 2.56s 2.59s p=0.931 n=6
Bind Time 0.99s (± 0.52%) 1.00s (± 0.84%) ~ 0.99s 1.01s p=0.070 n=6
Check Time 16.56s (± 0.39%) 16.67s (± 0.61%) ~ 16.51s 16.78s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.13s (± 0.33%) 20.23s (± 0.52%) ~ 20.08s 20.36s p=0.078 n=6
xstate - node (v18.15.0, x64)
Memory used 512,638k (± 0.01%) 512,630k (± 0.02%) ~ 512,533k 512,730k p=0.936 n=6
Parse Time 3.27s (± 0.45%) 3.27s (± 0.36%) ~ 3.25s 3.28s p=1.000 n=6
Bind Time 1.55s (± 0.48%) 1.55s (± 0.26%) ~ 1.54s 1.55s p=0.389 n=6
Check Time 2.87s (± 1.44%) 2.86s (± 0.76%) ~ 2.84s 2.90s p=0.629 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=0.405 n=6
Total Time 7.77s (± 0.47%) 7.76s (± 0.28%) ~ 7.74s 7.80s p=0.258 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,374ms (± 1.43%) 2,351ms (± 0.86%) ~ 2,322ms 2,381ms p=0.336 n=6
Req 2 - geterr 5,364ms (± 1.52%) 5,437ms (± 1.58%) ~ 5,344ms 5,520ms p=0.173 n=6
Req 3 - references 328ms (± 1.28%) 328ms (± 1.33%) ~ 324ms 335ms p=1.000 n=6
Req 4 - navto 277ms (± 1.36%) 277ms (± 0.81%) ~ 274ms 279ms p=0.746 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 84ms (± 8.24%) 84ms (± 8.09%) ~ 75ms 91ms p=0.616 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,474ms (± 1.37%) 2,468ms (± 1.40%) ~ 2,402ms 2,498ms p=0.423 n=6
Req 2 - geterr 4,135ms (± 1.83%) 4,158ms (± 2.02%) ~ 4,099ms 4,272ms p=0.378 n=6
Req 3 - references 338ms (± 1.62%) 341ms (± 1.71%) ~ 333ms 347ms p=0.325 n=6
Req 4 - navto 283ms (± 0.63%) 284ms (± 0.41%) ~ 282ms 285ms p=0.458 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 82ms (± 7.61%) 84ms (± 7.48%) ~ 76ms 90ms p=0.498 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,590ms (± 0.41%) 2,595ms (± 0.67%) ~ 2,572ms 2,612ms p=0.378 n=6
Req 2 - geterr 1,705ms (± 1.75%) 1,679ms (± 1.07%) ~ 1,658ms 1,709ms p=0.128 n=6
Req 3 - references 109ms (± 7.48%) 110ms (± 8.14%) ~ 105ms 128ms p=0.932 n=6
Req 4 - navto 359ms (± 0.18%) 359ms (± 0.38%) ~ 357ms 361ms p=0.236 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 302ms (± 0.97%) 307ms (± 0.56%) +5ms (+ 1.66%) 305ms 309ms p=0.014 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.04ms (± 0.21%) 152.04ms (± 0.18%) ~ 150.98ms 155.38ms p=0.877 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.90ms (± 0.16%) 227.73ms (± 0.19%) -0.17ms (- 0.07%) 226.40ms 234.08ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.85ms (± 0.15%) 228.98ms (± 0.20%) +0.12ms (+ 0.05%) 227.22ms 239.82ms p=0.013 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 228.86ms (± 0.16%) 228.85ms (± 0.16%) ~ 227.40ms 232.04ms p=0.642 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Hm, not very good, probably because if we're already willing to pay the cost to do relations, it's not going to be very expensive to go through simpleTypeRelated or whatever. But in the other place, we're guaranteed to call relation 4 times when the types aren't related, which is expensive on large unions.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/56072/merge:

Everything looks good!

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 19, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at f993810. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,651k (± 0.01%) 295,652k (± 0.01%) ~ 295,626k 295,680k p=0.689 n=6
Parse Time 2.66s (± 0.00%) 2.66s (± 0.24%) ~ 2.65s 2.67s p=1.000 n=6
Bind Time 0.83s (± 0.90%) 0.83s (± 1.08%) ~ 0.82s 0.84s p=0.798 n=6
Check Time 8.19s (± 0.30%) 8.19s (± 0.31%) ~ 8.16s 8.22s p=1.000 n=6
Emit Time 7.11s (± 0.31%) 7.11s (± 0.23%) ~ 7.09s 7.14s p=1.000 n=6
Total Time 18.79s (± 0.05%) 18.80s (± 0.20%) ~ 18.73s 18.84s p=0.164 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,535k (± 1.28%) 192,979k (± 1.27%) ~ 191,508k 197,462k p=0.873 n=6
Parse Time 1.36s (± 0.72%) 1.36s (± 0.46%) ~ 1.35s 1.37s p=1.000 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.36s (± 0.42%) 9.35s (± 0.41%) ~ 9.31s 9.40s p=0.936 n=6
Emit Time 2.62s (± 0.59%) 2.60s (± 1.36%) ~ 2.54s 2.64s p=0.225 n=6
Total Time 14.06s (± 0.22%) 14.03s (± 0.36%) ~ 13.98s 14.11s p=0.296 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,462k (± 0.01%) 347,459k (± 0.00%) ~ 347,450k 347,476k p=0.809 n=6
Parse Time 2.48s (± 0.47%) 2.48s (± 0.61%) ~ 2.47s 2.51s p=1.000 n=6
Bind Time 0.93s (± 0.55%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.174 n=6
Check Time 6.90s (± 0.39%) 6.89s (± 0.22%) ~ 6.87s 6.91s p=0.408 n=6
Emit Time 4.06s (± 0.62%) 4.06s (± 0.19%) ~ 4.05s 4.07s p=1.000 n=6
Total Time 14.38s (± 0.32%) 14.36s (± 0.13%) ~ 14.32s 14.37s p=0.290 n=6
TFS - node (v18.15.0, x64)
Memory used 302,829k (± 0.01%) 302,840k (± 0.01%) ~ 302,819k 302,859k p=0.298 n=6
Parse Time 2.01s (± 1.07%) 2.00s (± 0.88%) ~ 1.99s 2.04s p=0.317 n=6
Bind Time 1.00s (± 0.81%) 1.01s (± 1.20%) ~ 0.99s 1.02s p=0.652 n=6
Check Time 6.33s (± 0.12%) 6.31s (± 0.23%) -0.02s (- 0.26%) 6.29s 6.33s p=0.046 n=6
Emit Time 3.59s (± 0.42%) 3.59s (± 0.65%) ~ 3.56s 3.62s p=0.871 n=6
Total Time 12.94s (± 0.18%) 12.91s (± 0.28%) ~ 12.88s 12.97s p=0.195 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,295k (± 0.00%) 511,295k (± 0.01%) ~ 511,250k 511,321k p=0.688 n=6
Parse Time 2.66s (± 0.19%) 2.65s (± 0.74%) ~ 2.62s 2.67s p=0.797 n=6
Bind Time 1.00s (± 1.37%) 1.00s (± 0.54%) ~ 1.00s 1.01s p=0.154 n=6
Check Time 17.27s (± 0.36%) 17.25s (± 0.31%) ~ 17.19s 17.35s p=0.573 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.93s (± 0.33%) 20.91s (± 0.29%) ~ 20.82s 21.01s p=0.470 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,695,944k (± 0.00%) 1,695,962k (± 0.00%) ~ 1,695,913k 1,696,005k p=0.575 n=6
Parse Time 6.53s (± 0.40%) 6.54s (± 0.60%) ~ 6.50s 6.61s p=0.747 n=6
Bind Time 2.35s (± 0.35%) 2.34s (± 0.50%) ~ 2.33s 2.36s p=0.066 n=6
Check Time 55.93s (± 3.01%) 55.37s (± 0.36%) ~ 55.16s 55.69s p=0.630 n=6
Emit Time 0.16s (± 0.00%) 0.16s (± 0.00%) ~ 0.16s 0.16s p=1.000 n=6
Total Time 64.97s (± 2.61%) 64.42s (± 0.37%) ~ 64.18s 64.83s p=0.748 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,413,387k (± 0.03%) 2,413,106k (± 0.02%) ~ 2,412,589k 2,413,969k p=0.298 n=6
Parse Time 4.95s (± 1.25%) 4.96s (± 0.88%) ~ 4.92s 5.02s p=0.521 n=6
Bind Time 1.87s (± 0.53%) 1.88s (± 0.52%) ~ 1.87s 1.89s p=0.151 n=6
Check Time 33.39s (± 0.17%) 33.26s (± 0.18%) -0.13s (- 0.39%) 33.20s 33.34s p=0.013 n=6
Emit Time 2.66s (± 1.75%) 2.73s (± 1.32%) +0.08s (+ 2.88%) 2.69s 2.79s p=0.025 n=6
Total Time 42.90s (± 0.29%) 42.85s (± 0.10%) ~ 42.78s 42.89s p=0.230 n=6
self-compiler - node (v18.15.0, x64)
Memory used 419,682k (± 0.01%) 419,744k (± 0.01%) +61k (+ 0.01%) 419,693k 419,776k p=0.031 n=6
Parse Time 2.77s (± 3.57%) 2.75s (± 2.90%) ~ 2.68s 2.85s p=1.000 n=6
Bind Time 1.12s (± 6.38%) 1.14s (± 6.74%) ~ 1.07s 1.22s p=0.864 n=6
Check Time 15.09s (± 0.27%) 15.09s (± 0.41%) ~ 15.01s 15.19s p=1.000 n=6
Emit Time 1.17s (± 1.37%) 1.16s (± 2.08%) ~ 1.12s 1.19s p=0.805 n=6
Total Time 20.14s (± 0.24%) 20.14s (± 0.33%) ~ 20.03s 20.23s p=1.000 n=6
vscode - node (v18.15.0, x64)
Memory used 2,807,322k (± 0.00%) 2,807,291k (± 0.00%) ~ 2,807,192k 2,807,361k p=0.689 n=6
Parse Time 10.63s (± 0.44%) 10.63s (± 0.40%) ~ 10.59s 10.69s p=1.000 n=6
Bind Time 3.38s (± 0.22%) 3.40s (± 0.71%) ~ 3.36s 3.43s p=0.118 n=6
Check Time 59.72s (± 0.23%) 59.77s (± 0.44%) ~ 59.47s 60.10s p=1.000 n=6
Emit Time 16.17s (± 0.49%) 16.19s (± 0.34%) ~ 16.12s 16.26s p=0.689 n=6
Total Time 89.90s (± 0.07%) 89.99s (± 0.24%) ~ 89.77s 90.33s p=0.810 n=6
webpack - node (v18.15.0, x64)
Memory used 392,386k (± 0.02%) 392,418k (± 0.03%) ~ 392,307k 392,614k p=0.575 n=6
Parse Time 3.05s (± 0.72%) 3.06s (± 0.60%) ~ 3.04s 3.08s p=0.511 n=6
Bind Time 1.40s (± 0.45%) 1.41s (± 0.39%) ~ 1.40s 1.41s p=0.201 n=6
Check Time 14.01s (± 0.40%) 14.01s (± 0.46%) ~ 13.95s 14.13s p=0.936 n=6
Emit Time 0.00s (±244.70%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=0.405 n=6
Total Time 18.46s (± 0.36%) 18.47s (± 0.34%) ~ 18.44s 18.60s p=0.868 n=6
xstate - node (v18.15.0, x64)
Memory used 513,429k (± 0.02%) 513,432k (± 0.01%) ~ 513,337k 513,494k p=0.471 n=6
Parse Time 3.28s (± 0.16%) 3.28s (± 0.45%) ~ 3.26s 3.30s p=1.000 n=6
Bind Time 1.54s (± 0.35%) 1.54s (± 0.27%) ~ 1.53s 1.54s p=0.054 n=6
Check Time 2.85s (± 0.52%) 2.85s (± 0.75%) ~ 2.83s 2.89s p=0.683 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.76s (± 0.30%) 7.75s (± 0.19%) ~ 7.73s 7.77s p=0.460 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,342ms (± 0.70%) 2,353ms (± 0.89%) ~ 2,318ms 2,372ms p=0.297 n=6
Req 2 - geterr 5,529ms (± 1.55%) 5,529ms (± 1.61%) ~ 5,430ms 5,633ms p=0.810 n=6
Req 3 - references 323ms (± 0.36%) 325ms (± 1.13%) ~ 320ms 330ms p=0.121 n=6
Req 4 - navto 274ms (± 1.24%) 273ms (± 1.38%) ~ 270ms 278ms p=0.935 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 88ms (± 7.36%) 90ms (± 7.31%) ~ 79ms 95ms p=0.743 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,487ms (± 0.69%) 2,488ms (± 0.86%) ~ 2,460ms 2,509ms p=0.872 n=6
Req 2 - geterr 4,212ms (± 1.70%) 4,208ms (± 1.91%) ~ 4,124ms 4,288ms p=0.378 n=6
Req 3 - references 331ms (± 0.80%) 337ms (± 1.69%) +6ms (+ 1.81%) 331ms 343ms p=0.044 n=6
Req 4 - navto 285ms (± 1.28%) 284ms (± 0.26%) ~ 283ms 285ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 77ms (± 3.34%) 84ms (± 7.85%) ~ 77ms 90ms p=0.105 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,601ms (± 0.29%) 2,604ms (± 0.59%) ~ 2,584ms 2,626ms p=0.688 n=6
Req 2 - geterr 1,704ms (± 2.75%) 1,705ms (± 2.19%) ~ 1,669ms 1,741ms p=0.688 n=6
Req 3 - references 124ms (± 6.28%) 124ms (± 5.47%) ~ 110ms 128ms p=0.565 n=6
Req 4 - navto 370ms (± 0.22%) 371ms (± 0.33%) ~ 370ms 373ms p=0.126 n=6
Req 5 - completionInfo count 2,078 (± 0.00%) 2,078 (± 0.00%) ~ 2,078 2,078 p=1.000 n=6
Req 5 - completionInfo 308ms (± 1.73%) 308ms (± 2.38%) ~ 297ms 317ms p=0.936 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 153.87ms (± 0.22%) 153.95ms (± 0.22%) +0.09ms (+ 0.06%) 152.60ms 160.27ms p=0.001 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.94ms (± 0.16%) 229.78ms (± 0.17%) -0.16ms (- 0.07%) 228.53ms 236.76ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 231.55ms (± 0.19%) 231.46ms (± 0.20%) -0.09ms (- 0.04%) 229.88ms 237.82ms p=0.033 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 231.18ms (± 0.19%) 231.30ms (± 0.18%) +0.12ms (+ 0.05%) 229.55ms 234.45ms p=0.005 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey mentioned this pull request Jul 1, 2024
Closed
@sandersn
Copy link
Member

sandersn commented Apr 3, 2025

@jakebailey did the results indicate that this is worth pursuing? It is worth keeping open?
A quick glance at the latest perf run didn't show anything substantial.

@jakebailey
Copy link
Member Author

You can see the relevant benchmark in the original post; our current test suite doesn't have a case like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weswigham weswigham weswigham left review comments

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jakebailey @typescript-bot @sandersn @weswigham