Skip to content

Replace disk path operations to use fileNames instead of path #56476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 28, 2023

Conversation

sheetalkamat
Copy link
Member

@sheetalkamat sheetalkamat commented Nov 21, 2023

#55427 reported that the new files are not detected and per @gluxon investigation it turned out that the case was when tsserver was on case-insensitive volume while the user code was on the case sensitive volume. While changing complete architecture of useCaseSensitive and getCanonicalName is huge change that needs to percolate everywhere, trying to use file names in all the host operations and using canonical path for comparing (though that's not perfect and bar.ts and Bar.ts will be treated as same file) is minimal change that can help with some of the scenarios. While you may get wrong casing in say some of the auto imports but at least you should get completions etc just like when you open tsserver.

@sheetalkamat sheetalkamat changed the title [wip] Replace disk path operations to use fileNames instead of path Replace disk path operations to use fileNames instead of path Nov 21, 2023
@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Nov 21, 2023
@sheetalkamat
Copy link
Member Author

@gluxon can you please try this build and see if this helps

@sheetalkamat
Copy link
Member Author

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the tarball bundle task on this PR at 3cc03d4. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Hey @sheetalkamat, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/158728/artifacts?artifactName=tgz&fileId=F31A95D2779F62EF9B23FA910165385B3CF1F4BAB98F161AB22C96EC489C64A702&fileName=/typescript-5.4.0-insiders.20231121.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@sheetalkamat
Copy link
Member Author

@typescript-bot test this
@typescript-bot test top100
@typescript-bot test tsserver top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the diff-based top-repos suite (tsserver) on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the diff-based top-repos suite on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the diff-based user code test suite on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the diff-based user code test suite (tsserver) on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the parallelized Definitely Typed test suite on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the regular perf test suite on this PR at 3cc03d4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,182k (± 0.01%) 295,199k (± 0.01%) ~ 295,165k 295,230k p=0.378 n=6
Parse Time 2.64s (± 0.34%) 2.64s (± 0.48%) ~ 2.62s 2.65s p=0.933 n=6
Bind Time 0.83s (± 0.66%) 0.83s (± 0.76%) ~ 0.82s 0.84s p=0.201 n=6
Check Time 8.06s (± 0.18%) 8.03s (± 0.31%) ~ 8.00s 8.06s p=0.072 n=6
Emit Time 7.06s (± 0.24%) 7.08s (± 0.22%) ~ 7.06s 7.09s p=0.114 n=6
Total Time 18.58s (± 0.13%) 18.58s (± 0.14%) ~ 18.54s 18.61s p=1.000 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 194,537k (± 1.29%) 192,652k (± 1.55%) ~ 190,702k 196,539k p=0.093 n=6
Parse Time 1.36s (± 1.29%) 1.36s (± 1.00%) ~ 1.35s 1.38s p=0.563 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.18s (± 0.33%) 9.20s (± 0.65%) ~ 9.12s 9.27s p=0.520 n=6
Emit Time 2.63s (± 0.31%) 2.63s (± 0.96%) ~ 2.60s 2.67s p=0.864 n=6
Total Time 13.88s (± 0.33%) 13.91s (± 0.49%) ~ 13.83s 14.00s p=0.517 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,345k (± 0.00%) 347,344k (± 0.00%) ~ 347,328k 347,358k p=0.936 n=6
Parse Time 2.46s (± 0.48%) 2.46s (± 0.36%) ~ 2.45s 2.47s p=0.672 n=6
Bind Time 0.92s (± 0.59%) 0.93s (± 0.56%) ~ 0.92s 0.93s p=0.640 n=6
Check Time 6.94s (± 0.65%) 6.91s (± 0.31%) ~ 6.88s 6.94s p=0.260 n=6
Emit Time 4.05s (± 0.33%) 4.04s (± 0.62%) ~ 4.01s 4.08s p=0.464 n=6
Total Time 14.37s (± 0.34%) 14.33s (± 0.18%) ~ 14.31s 14.38s p=0.086 n=6
TFS - node (v18.15.0, x64)
Memory used 302,639k (± 0.01%) 302,652k (± 0.01%) ~ 302,618k 302,710k p=0.378 n=6
Parse Time 1.99s (± 0.32%) 2.00s (± 0.26%) +0.01s (+ 0.67%) 2.00s 2.01s p=0.009 n=6
Bind Time 1.00s (± 1.17%) 0.99s (± 0.52%) ~ 0.99s 1.00s p=0.191 n=6
Check Time 6.26s (± 0.41%) 6.29s (± 0.53%) ~ 6.24s 6.33s p=0.171 n=6
Emit Time 3.58s (± 0.67%) 3.58s (± 0.52%) ~ 3.55s 3.60s p=0.808 n=6
Total Time 12.84s (± 0.35%) 12.87s (± 0.27%) ~ 12.83s 12.92s p=0.293 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,564k (± 0.01%) 470,735k (± 0.01%) +170k (+ 0.04%) 470,680k 470,788k p=0.005 n=6
Parse Time 2.58s (± 0.32%) 2.60s (± 0.45%) +0.02s (+ 0.91%) 2.58s 2.61s p=0.010 n=6
Bind Time 0.99s (± 1.49%) 0.98s (± 1.19%) ~ 0.97s 1.00s p=0.323 n=6
Check Time 16.71s (± 0.27%) 16.71s (± 0.48%) ~ 16.61s 16.84s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.28s (± 0.18%) 20.29s (± 0.45%) ~ 20.17s 20.44s p=0.809 n=6
xstate - node (v18.15.0, x64)
Memory used 511,980k (± 0.01%) 512,205k (± 0.00%) +225k (+ 0.04%) 512,188k 512,224k p=0.005 n=6
Parse Time 3.27s (± 0.32%) 3.29s (± 0.12%) +0.02s (+ 0.56%) 3.29s 3.30s p=0.003 n=6
Bind Time 1.54s (± 0.49%) 1.55s (± 0.53%) ~ 1.54s 1.56s p=0.383 n=6
Check Time 2.79s (± 0.49%) 2.81s (± 0.52%) +0.02s (+ 0.90%) 2.80s 2.84s p=0.022 n=6
Emit Time 0.08s (± 6.19%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=0.174 n=6
Total Time 7.68s (± 0.19%) 7.72s (± 0.15%) +0.04s (+ 0.52%) 7.70s 7.73s p=0.006 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,352ms (± 0.54%) 2,359ms (± 0.88%) ~ 2,336ms 2,391ms p=0.810 n=6
Req 2 - geterr 5,422ms (± 1.59%) 5,417ms (± 1.63%) ~ 5,281ms 5,487ms p=0.936 n=6
Req 3 - references 328ms (± 1.42%) 328ms (± 2.03%) ~ 323ms 341ms p=1.000 n=6
Req 4 - navto 276ms (± 1.27%) 276ms (± 1.14%) ~ 273ms 280ms p=0.933 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 87ms (± 5.40%) 86ms (± 5.36%) ~ 81ms 91ms p=0.681 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,507ms (± 0.46%) 2,507ms (± 0.76%) ~ 2,481ms 2,528ms p=0.688 n=6
Req 2 - geterr 4,038ms (± 1.73%) 4,041ms (± 1.56%) ~ 4,008ms 4,170ms p=0.423 n=6
Req 3 - references 341ms (± 1.48%) 341ms (± 1.65%) ~ 333ms 346ms p=1.000 n=6
Req 4 - navto 284ms (± 0.48%) 283ms (± 0.36%) ~ 282ms 284ms p=0.931 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 88ms (± 5.47%) 86ms (± 6.89%) ~ 78ms 90ms p=1.000 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,594ms (± 0.68%) 2,622ms (± 0.29%) +28ms (+ 1.09%) 2,608ms 2,629ms p=0.013 n=6
Req 2 - geterr 1,657ms (± 1.72%) 1,679ms (± 2.27%) ~ 1,606ms 1,712ms p=0.230 n=6
Req 3 - references 115ms (± 8.78%) 118ms (± 6.05%) ~ 105ms 124ms p=0.572 n=6
Req 4 - navto 366ms (± 0.66%) 364ms (± 1.19%) ~ 358ms 369ms p=0.572 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 309ms (± 1.24%) 316ms (± 1.65%) ~ 308ms 322ms p=0.054 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.71ms (± 0.19%) 152.80ms (± 0.18%) +0.10ms (+ 0.06%) 151.55ms 156.60ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 228.15ms (± 0.16%) 228.01ms (± 0.17%) -0.15ms (- 0.06%) 226.54ms 233.15ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 229.40ms (± 0.18%) 229.61ms (± 0.17%) +0.21ms (+ 0.09%) 228.18ms 234.92ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 229.13ms (± 0.16%) 229.14ms (± 0.20%) ~ 227.60ms 238.02ms p=0.525 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the user test suite comparing main and refs/pull/56476/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the user test suite comparing main and refs/pull/56476/merge:

There were infrastructure failures potentially unrelated to your change:

  • 2 instances of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

puppeteer

packages/browsers/test/src/tsconfig.json

@typescript-bot
Copy link
Collaborator

Hey @sheetalkamat, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the top-repos suite comparing main and refs/pull/56476/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the top-repos suite comparing main and refs/pull/56476/merge:

Something interesting changed - please have a look.

Details

Server exited prematurely with code unknown and signal SIGABRT

Server exited prematurely with code unknown and signal SIGABRT

Affected repos

calcom/cal.com Raw error text: RepoResults7/calcom.cal.com.rawError.txt in the artifact folder

Last few requests

{"seq":941,"type":"request","command":"navto","arguments":{"searchValue":"a1R","maxResultCount":256}}
{"seq":942,"type":"request","command":"updateOpen","arguments":{"changedFiles":[{"fileName":"@PROJECT_ROOT@/apps/swagger/pages/_app.tsx","textChanges":[{"newText":" //comment","start":{"line":1,"offset":42},"end":{"line":1,"offset":42}}]}],"closedFiles":[],"openFiles":[]}}
{"seq":943,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":["@PROJECT_ROOT@/apps/api/test/lib/middleware/withMiddleware.test.ts"],"openFiles":[]}}
{"seq":944,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/apps/web/components/apps/AdditionalCalendarSelector.tsx","projectRootPath":"@PROJECT_ROOT@"}]}}

Repro steps

  1. git clone https://github.com/calcom/cal.com --recurse-submodules
  2. In dir cal.com, run git reset --hard 48dde246e92fd2ed94c84bf39389078ab9ec639a
  3. In dir cal.com, run yarn install --no-immutable --mode=skip-build
  4. Back in the initial folder, download RepoResults7/calcom.cal.com.replay.txt from the artifact folder
  5. npm install --no-save @typescript/server-replay
  6. npx tsreplay ./cal.com ./calcom.cal.com.replay.txt path/to/tsserver.js
  7. npx tsreplay --help to learn about helpful switches for debugging, logging, etc

@sheetalkamat
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 21, 2023

Heya @sheetalkamat, I've started to run the regular perf test suite on this PR at 5468827. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,191k (± 0.01%) 295,174k (± 0.01%) ~ 295,121k 295,221k p=0.378 n=6
Parse Time 2.64s (± 0.15%) 2.65s (± 0.40%) ~ 2.63s 2.66s p=0.528 n=6
Bind Time 0.83s (± 1.01%) 0.83s (± 1.18%) ~ 0.82s 0.84s p=0.256 n=6
Check Time 8.04s (± 0.37%) 8.04s (± 0.19%) ~ 8.02s 8.06s p=0.366 n=6
Emit Time 7.07s (± 0.25%) 7.06s (± 0.20%) ~ 7.04s 7.08s p=0.413 n=6
Total Time 18.58s (± 0.18%) 18.58s (± 0.14%) ~ 18.56s 18.63s p=0.739 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 194,618k (± 1.56%) 193,118k (± 1.53%) ~ 190,656k 196,707k p=0.689 n=6
Parse Time 1.36s (± 1.27%) 1.37s (± 0.85%) ~ 1.35s 1.38s p=0.318 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.16s (± 0.20%) 9.20s (± 0.32%) +0.04s (+ 0.42%) 9.17s 9.24s p=0.044 n=6
Emit Time 2.62s (± 0.38%) 2.62s (± 0.71%) ~ 2.59s 2.64s p=0.391 n=6
Total Time 13.86s (± 0.11%) 13.91s (± 0.21%) +0.05s (+ 0.37%) 13.87s 13.95s p=0.012 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,364k (± 0.00%) 347,347k (± 0.00%) ~ 347,328k 347,369k p=0.054 n=6
Parse Time 2.45s (± 0.49%) 2.45s (± 0.60%) ~ 2.43s 2.47s p=0.934 n=6
Bind Time 0.92s (± 0.59%) 0.92s (± 0.44%) ~ 0.92s 0.93s p=0.282 n=6
Check Time 6.91s (± 0.64%) 6.89s (± 0.35%) ~ 6.86s 6.93s p=0.809 n=6
Emit Time 4.04s (± 0.43%) 4.04s (± 0.37%) ~ 4.02s 4.06s p=0.871 n=6
Total Time 14.33s (± 0.34%) 14.31s (± 0.20%) ~ 14.27s 14.36s p=0.571 n=6
TFS - node (v18.15.0, x64)
Memory used 302,634k (± 0.00%) 302,634k (± 0.01%) ~ 302,605k 302,673k p=0.810 n=6
Parse Time 2.02s (± 0.75%) 2.00s (± 0.80%) ~ 1.98s 2.02s p=0.140 n=6
Bind Time 1.00s (± 1.36%) 1.00s (± 1.03%) ~ 0.99s 1.02s p=0.933 n=6
Check Time 6.27s (± 0.33%) 6.26s (± 0.62%) ~ 6.21s 6.31s p=0.420 n=6
Emit Time 3.59s (± 0.33%) 3.58s (± 0.84%) ~ 3.54s 3.63s p=0.195 n=6
Total Time 12.88s (± 0.19%) 12.84s (± 0.29%) ~ 12.79s 12.89s p=0.091 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,564k (± 0.01%) 470,748k (± 0.01%) +184k (+ 0.04%) 470,689k 470,783k p=0.005 n=6
Parse Time 2.56s (± 0.48%) 2.60s (± 0.31%) +0.03s (+ 1.23%) 2.58s 2.60s p=0.005 n=6
Bind Time 0.99s (± 0.99%) 0.99s (± 1.28%) ~ 0.97s 1.00s p=1.000 n=6
Check Time 16.75s (± 0.47%) 16.73s (± 0.40%) ~ 16.66s 16.82s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.30s (± 0.37%) 20.32s (± 0.38%) ~ 20.24s 20.41s p=0.748 n=6
xstate - node (v18.15.0, x64)
Memory used 511,966k (± 0.01%) 512,192k (± 0.01%) +226k (+ 0.04%) 512,144k 512,261k p=0.005 n=6
Parse Time 3.27s (± 0.23%) 3.29s (± 0.38%) +0.02s (+ 0.56%) 3.28s 3.31s p=0.015 n=6
Bind Time 1.54s (± 0.41%) 1.54s (± 0.26%) ~ 1.54s 1.55s p=0.673 n=6
Check Time 2.81s (± 1.38%) 2.79s (± 0.30%) ~ 2.79s 2.81s p=0.414 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.70s (± 0.48%) 7.70s (± 0.29%) ~ 7.68s 7.74s p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,368ms (± 0.49%) 2,363ms (± 0.48%) ~ 2,343ms 2,376ms p=0.936 n=6
Req 2 - geterr 5,370ms (± 1.55%) 5,375ms (± 1.39%) ~ 5,311ms 5,475ms p=0.575 n=6
Req 3 - references 327ms (± 1.13%) 330ms (± 1.15%) ~ 326ms 334ms p=0.277 n=6
Req 4 - navto 278ms (± 1.00%) 277ms (± 1.22%) ~ 273ms 280ms p=0.933 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 83ms (± 4.93%) 83ms (± 6.37%) ~ 79ms 90ms p=0.730 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,486ms (± 1.90%) 2,493ms (± 1.09%) ~ 2,445ms 2,518ms p=0.936 n=6
Req 2 - geterr 4,065ms (± 1.66%) 4,066ms (± 1.73%) ~ 4,001ms 4,160ms p=0.936 n=6
Req 3 - references 342ms (± 1.86%) 342ms (± 1.54%) ~ 334ms 346ms p=1.000 n=6
Req 4 - navto 284ms (± 0.39%) 283ms (± 0.29%) ~ 282ms 284ms p=0.197 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 85ms (± 6.61%) 86ms (± 7.11%) ~ 77ms 90ms p=1.000 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,602ms (± 0.42%) 2,615ms (± 0.18%) +13ms (+ 0.51%) 2,609ms 2,621ms p=0.045 n=6
Req 2 - geterr 1,659ms (± 2.73%) 1,669ms (± 2.05%) ~ 1,606ms 1,697ms p=0.575 n=6
Req 3 - references 118ms (± 7.14%) 113ms (± 9.16%) ~ 102ms 122ms p=0.389 n=6
Req 4 - navto 366ms (± 0.63%) 364ms (± 0.95%) ~ 358ms 367ms p=0.323 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 310ms (± 1.92%) 314ms (± 1.75%) ~ 305ms 318ms p=0.195 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.77ms (± 0.20%) 152.72ms (± 0.18%) ~ 151.43ms 154.72ms p=0.295 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 228.01ms (± 0.14%) 227.79ms (± 0.17%) -0.21ms (- 0.09%) 226.30ms 233.65ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 229.38ms (± 0.19%) 229.50ms (± 0.19%) +0.11ms (+ 0.05%) 227.96ms 235.71ms p=0.003 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 229.11ms (± 0.19%) 229.17ms (± 0.18%) ~ 227.84ms 236.97ms p=0.071 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@sheetalkamat
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 27, 2023

Heya @sheetalkamat, I've started to run the regular perf test suite on this PR at aed1e9b. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,191k (± 0.01%) 295,165k (± 0.01%) ~ 295,103k 295,223k p=0.229 n=6
Parse Time 2.65s (± 0.21%) 2.64s (± 0.20%) ~ 2.64s 2.65s p=0.640 n=6
Bind Time 0.82s (± 0.63%) 0.82s (± 1.20%) ~ 0.81s 0.84s p=0.504 n=6
Check Time 8.04s (± 0.07%) 8.03s (± 0.34%) ~ 7.99s 8.05s p=0.663 n=6
Emit Time 7.07s (± 0.29%) 7.05s (± 0.18%) ~ 7.03s 7.06s p=0.059 n=6
Total Time 18.59s (± 0.11%) 18.54s (± 0.16%) -0.05s (- 0.26%) 18.50s 18.58s p=0.019 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 194,582k (± 1.55%) 194,545k (± 1.54%) ~ 190,663k 196,535k p=0.936 n=6
Parse Time 1.36s (± 1.01%) 1.36s (± 0.72%) ~ 1.35s 1.37s p=0.457 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.15s (± 0.63%) 9.15s (± 0.48%) ~ 9.10s 9.21s p=0.871 n=6
Emit Time 2.61s (± 0.92%) 2.62s (± 0.52%) ~ 2.60s 2.64s p=0.415 n=6
Total Time 13.83s (± 0.39%) 13.85s (± 0.28%) ~ 13.80s 13.89s p=0.685 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,363k (± 0.00%) 347,360k (± 0.01%) ~ 347,335k 347,378k p=0.936 n=6
Parse Time 2.47s (± 0.30%) 2.45s (± 0.75%) ~ 2.43s 2.48s p=0.050 n=6
Bind Time 0.92s (± 0.69%) 0.92s (± 0.82%) ~ 0.91s 0.93s p=0.718 n=6
Check Time 6.90s (± 0.77%) 6.89s (± 0.34%) ~ 6.86s 6.92s p=0.748 n=6
Emit Time 4.06s (± 0.24%) 4.05s (± 0.30%) ~ 4.03s 4.06s p=0.059 n=6
Total Time 14.35s (± 0.44%) 14.31s (± 0.23%) ~ 14.26s 14.36s p=0.258 n=6
TFS - node (v18.15.0, x64)
Memory used 302,627k (± 0.00%) 302,635k (± 0.01%) ~ 302,600k 302,719k p=1.000 n=6
Parse Time 2.00s (± 1.17%) 2.00s (± 1.16%) ~ 1.97s 2.02s p=0.934 n=6
Bind Time 1.00s (± 0.75%) 1.01s (± 1.46%) ~ 0.99s 1.02s p=0.365 n=6
Check Time 6.27s (± 0.62%) 6.28s (± 0.49%) ~ 6.23s 6.31s p=0.630 n=6
Emit Time 3.58s (± 0.41%) 3.59s (± 0.42%) ~ 3.56s 3.60s p=0.369 n=6
Total Time 12.84s (± 0.24%) 12.87s (± 0.46%) ~ 12.78s 12.92s p=0.296 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,595k (± 0.01%) 470,611k (± 0.02%) ~ 470,529k 470,712k p=0.810 n=6
Parse Time 2.57s (± 0.40%) 2.57s (± 0.32%) ~ 2.57s 2.59s p=0.210 n=6
Bind Time 0.98s (± 1.19%) 0.99s (± 0.83%) ~ 0.98s 1.00s p=0.666 n=6
Check Time 16.79s (± 0.43%) 16.73s (± 0.31%) ~ 16.65s 16.79s p=0.171 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.34s (± 0.30%) 20.29s (± 0.25%) ~ 20.22s 20.36s p=0.226 n=6
xstate - node (v18.15.0, x64)
Memory used 511,994k (± 0.02%) 511,980k (± 0.01%) ~ 511,942k 512,008k p=0.810 n=6
Parse Time 3.28s (± 0.16%) 3.27s (± 0.16%) ~ 3.27s 3.28s p=0.311 n=6
Bind Time 1.54s (± 0.26%) 1.54s (± 0.33%) ~ 1.54s 1.55s p=0.595 n=6
Check Time 2.81s (± 1.16%) 2.80s (± 0.87%) ~ 2.77s 2.83s p=0.466 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.174 n=6
Total Time 7.71s (± 0.39%) 7.70s (± 0.30%) ~ 7.67s 7.72s p=0.329 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,357ms (± 0.39%) 2,368ms (± 0.57%) ~ 2,350ms 2,379ms p=0.199 n=6
Req 2 - geterr 5,414ms (± 1.18%) 5,402ms (± 1.62%) ~ 5,303ms 5,508ms p=0.810 n=6
Req 3 - references 328ms (± 1.34%) 327ms (± 0.42%) ~ 325ms 329ms p=0.737 n=6
Req 4 - navto 276ms (± 1.25%) 276ms (± 1.33%) ~ 273ms 280ms p=1.000 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 86ms (± 5.45%) 84ms (± 7.37%) ~ 76ms 90ms p=0.452 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,486ms (± 1.72%) 2,491ms (± 1.27%) ~ 2,442ms 2,525ms p=0.810 n=6
Req 2 - geterr 4,057ms (± 1.83%) 4,063ms (± 1.98%) ~ 3,995ms 4,176ms p=0.689 n=6
Req 3 - references 341ms (± 1.56%) 341ms (± 1.45%) ~ 335ms 346ms p=0.869 n=6
Req 4 - navto 283ms (± 0.37%) 284ms (± 0.22%) +1ms (+ 0.47%) 283ms 285ms p=0.028 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 86ms (± 7.08%) 84ms (± 6.73%) ~ 77ms 90ms p=0.558 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,594ms (± 0.79%) 2,596ms (± 0.30%) ~ 2,588ms 2,610ms p=0.630 n=6
Req 2 - geterr 1,658ms (± 2.36%) 1,677ms (± 0.83%) ~ 1,665ms 1,704ms p=0.688 n=6
Req 3 - references 114ms (± 9.41%) 115ms (± 8.79%) ~ 102ms 124ms p=0.872 n=6
Req 4 - navto 365ms (± 0.56%) 364ms (± 0.17%) ~ 363ms 365ms p=0.363 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 309ms (± 1.90%) 311ms (± 1.52%) ~ 302ms 315ms p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.79ms (± 0.20%) 152.76ms (± 0.19%) ~ 151.51ms 156.66ms p=0.412 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 228.11ms (± 0.17%) 227.90ms (± 0.16%) -0.21ms (- 0.09%) 226.67ms 234.28ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 229.64ms (± 0.17%) 229.69ms (± 0.19%) ~ 228.07ms 234.86ms p=0.272 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 229.46ms (± 0.18%) 229.61ms (± 0.18%) +0.15ms (+ 0.06%) 228.25ms 232.78ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@sheetalkamat sheetalkamat marked this pull request as ready for review November 27, 2023 23:47
Copy link
Member

@andrewbranch andrewbranch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m sure it would be incredibly disruptive, but it kind of feels like we would benefit from a FileName branded string type like Path so that a Path can’t be passed to a function that needs the name of an existing file with original casing.

@sheetalkamat
Copy link
Member Author

I’m sure it would be incredibly disruptive, but it kind of feels like we would benefit from a FileName branded string type like Path so that a Path can’t be passed to a function that needs the name of an existing file with original casing.

We have this somewhat in server with normalized path but not in absolute path and we probably want that distinction as well .. but that’s too much

@sheetalkamat sheetalkamat merged commit b6121e4 into main Nov 28, 2023
@sheetalkamat sheetalkamat deleted the perVolumeCasing branch November 28, 2023 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants