Skip to content

Stop pre-emptively creating directories #34521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 22, 2019

Conversation

amcasey
Copy link
Member

@amcasey amcasey commented Oct 16, 2019

Checking for directory existence is expensive and frequently indicates success. Instead of pre-emptively creating the directory containing a file to be written, attempt to create the file and only do the directory scaffolding if the write fails.

Appears to reduce file write time by 10-20% for a file-I/O heavy partner build.

Thanks to @rbuckton for the suggestion!

Checking for directory existence is expensive and frequently indicates
success.  Instead of pre-emptively creating the directory containing a
file to be written, attempt to create the file and only do the directory
scaffolding if the write fails.

Appears to reduce file write time by 10-20% for a file-I/O heavy partner
build.

Thanks to @rbuckton for the suggestion!
@amcasey
Copy link
Member Author

amcasey commented Oct 16, 2019

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 16, 2019

Heya @amcasey, I've started to run the perf test suite on this PR at b6659e5. You can monitor the build here. It should now contribute to this PR's status checks.

Update: The results are in!

@amcasey amcasey requested a review from rbuckton October 16, 2019 18:38
@typescript-bot
Copy link
Collaborator

@amcasey
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..34521

Metric master 34521 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 356,067k (± 0.03%) 356,345k (± 0.02%) +278k (+ 0.08%) 356,151k 356,480k
Parse Time 1.61s (± 0.62%) 1.61s (± 0.58%) +0.00s (+ 0.19%) 1.59s 1.63s
Bind Time 0.85s (± 0.76%) 0.85s (± 0.68%) +0.00s (+ 0.12%) 0.84s 0.87s
Check Time 4.49s (± 0.26%) 4.50s (± 0.38%) +0.01s (+ 0.20%) 4.46s 4.55s
Emit Time 5.28s (± 1.09%) 5.24s (± 0.84%) -0.05s (- 0.89%) 5.17s 5.37s
Total Time 12.23s (± 0.51%) 12.20s (± 0.39%) -0.03s (- 0.26%) 12.12s 12.34s
Monaco - node (v10.16.3, x64)
Memory used 366,969k (± 0.02%) 366,922k (± 0.02%) -46k (- 0.01%) 366,838k 367,120k
Parse Time 1.25s (± 0.56%) 1.25s (± 0.39%) +0.00s (+ 0.24%) 1.24s 1.26s
Bind Time 0.75s (± 0.60%) 0.75s (± 0.74%) 0.00s ( 0.00%) 0.74s 0.76s
Check Time 4.62s (± 0.60%) 4.62s (± 0.41%) -0.00s (- 0.00%) 4.59s 4.67s
Emit Time 2.94s (± 0.49%) 2.94s (± 0.92%) +0.00s (+ 0.07%) 2.89s 3.01s
Total Time 9.56s (± 0.30%) 9.57s (± 0.47%) +0.01s (+ 0.05%) 9.48s 9.67s
TFS - node (v10.16.3, x64)
Memory used 322,541k (± 0.01%) 322,482k (± 0.02%) -59k (- 0.02%) 322,311k 322,658k
Parse Time 0.95s (± 0.58%) 0.95s (± 0.79%) +0.00s (+ 0.21%) 0.94s 0.97s
Bind Time 0.71s (± 1.67%) 0.72s (± 1.24%) +0.00s (+ 0.56%) 0.69s 0.73s
Check Time 4.12s (± 0.46%) 4.12s (± 0.59%) +0.00s (+ 0.07%) 4.06s 4.16s
Emit Time 3.04s (± 0.84%) 3.03s (± 0.67%) -0.00s (- 0.10%) 2.99s 3.08s
Total Time 8.82s (± 0.40%) 8.82s (± 0.44%) +0.01s (+ 0.06%) 8.73s 8.91s
Angular - node (v12.1.0, x64)
Memory used 331,622k (± 0.04%) 331,917k (± 0.04%) +295k (+ 0.09%) 331,391k 332,072k
Parse Time 1.56s (± 0.49%) 1.56s (± 0.85%) +0.01s (+ 0.45%) 1.54s 1.60s
Bind Time 0.84s (± 0.86%) 0.84s (± 0.95%) 0.00s ( 0.00%) 0.83s 0.86s
Check Time 4.42s (± 0.47%) 4.43s (± 0.45%) +0.01s (+ 0.34%) 4.40s 4.49s
Emit Time 5.44s (± 1.27%) 5.45s (± 0.99%) +0.01s (+ 0.18%) 5.34s 5.58s
Total Time 12.25s (± 0.67%) 12.29s (± 0.65%) +0.03s (+ 0.26%) 12.18s 12.51s
Monaco - node (v12.1.0, x64)
Memory used 346,737k (± 0.01%) 346,714k (± 0.02%) -22k (- 0.01%) 346,508k 346,864k
Parse Time 1.21s (± 0.62%) 1.22s (± 0.77%) +0.01s (+ 0.74%) 1.21s 1.25s
Bind Time 0.73s (± 1.72%) 0.72s (± 0.83%) -0.00s (- 0.14%) 0.72s 0.74s
Check Time 4.46s (± 0.30%) 4.49s (± 0.56%) +0.03s (+ 0.74%) 4.44s 4.54s
Emit Time 2.99s (± 0.66%) 3.00s (± 0.61%) +0.01s (+ 0.23%) 2.97s 3.04s
Total Time 9.39s (± 0.34%) 9.44s (± 0.37%) +0.04s (+ 0.48%) 9.37s 9.52s
TFS - node (v12.1.0, x64)
Memory used 304,894k (± 0.02%) 304,877k (± 0.02%) -17k (- 0.01%) 304,734k 305,043k
Parse Time 0.94s (± 0.36%) 0.95s (± 0.74%) +0.01s (+ 0.85%) 0.94s 0.97s
Bind Time 0.68s (± 0.87%) 0.69s (± 1.35%) +0.00s (+ 0.44%) 0.68s 0.72s
Check Time 4.04s (± 0.36%) 4.06s (± 0.70%) +0.03s (+ 0.62%) 4.03s 4.17s
Emit Time 3.08s (± 0.86%) 3.10s (± 1.04%) +0.02s (+ 0.49%) 3.06s 3.21s
Total Time 8.75s (± 0.39%) 8.80s (± 0.73%) +0.05s (+ 0.58%) 8.73s 9.04s
Angular - node (v8.9.0, x64)
Memory used 350,851k (± 0.02%) 351,027k (± 0.02%) +176k (+ 0.05%) 350,895k 351,207k
Parse Time 2.10s (± 0.63%) 2.10s (± 0.36%) -0.01s (- 0.29%) 2.08s 2.12s
Bind Time 0.90s (± 0.56%) 0.89s (± 0.65%) -0.00s (- 0.22%) 0.88s 0.90s
Check Time 5.25s (± 0.41%) 5.26s (± 0.62%) +0.00s (+ 0.08%) 5.21s 5.36s
Emit Time 6.25s (± 1.23%) 6.27s (± 1.29%) +0.02s (+ 0.29%) 6.16s 6.57s
Total Time 14.50s (± 0.61%) 14.51s (± 0.74%) +0.01s (+ 0.06%) 14.33s 14.90s
Monaco - node (v8.9.0, x64)
Memory used 364,682k (± 0.01%) 364,675k (± 0.02%) -7k (- 0.00%) 364,575k 364,837k
Parse Time 1.56s (± 0.48%) 1.56s (± 0.48%) +0.01s (+ 0.45%) 1.55s 1.59s
Bind Time 0.92s (± 0.52%) 0.92s (± 0.84%) +0.00s (+ 0.22%) 0.91s 0.94s
Check Time 5.52s (± 0.51%) 5.54s (± 0.53%) +0.03s (+ 0.45%) 5.48s 5.61s
Emit Time 3.06s (± 0.94%) 3.04s (± 0.58%) -0.02s (- 0.78%) 2.98s 3.07s
Total Time 11.06s (± 0.45%) 11.07s (± 0.36%) +0.01s (+ 0.09%) 10.96s 11.16s
TFS - node (v8.9.0, x64)
Memory used 321,384k (± 0.01%) 321,379k (± 0.02%) -5k (- 0.00%) 321,258k 321,482k
Parse Time 1.27s (± 0.57%) 1.26s (± 0.60%) -0.00s (- 0.24%) 1.24s 1.28s
Bind Time 0.74s (± 0.51%) 0.73s (± 0.88%) -0.00s (- 0.41%) 0.72s 0.74s
Check Time 4.68s (± 0.60%) 4.71s (± 0.68%) +0.03s (+ 0.73%) 4.65s 4.78s
Emit Time 3.22s (± 0.72%) 3.21s (± 0.54%) -0.00s (- 0.12%) 3.18s 3.25s
Total Time 9.91s (± 0.33%) 9.92s (± 0.43%) +0.02s (+ 0.19%) 9.85s 10.03s
Angular - node (v8.9.0, x86)
Memory used 198,953k (± 0.02%) 199,146k (± 0.02%) +192k (+ 0.10%) 199,072k 199,255k
Parse Time 2.02s (± 0.57%) 2.02s (± 0.70%) 0.00s ( 0.00%) 1.99s 2.06s
Bind Time 1.01s (± 0.59%) 1.03s (± 0.90%) +0.01s (+ 1.28%) 1.01s 1.05s
Check Time 4.79s (± 0.61%) 4.78s (± 0.30%) -0.01s (- 0.17%) 4.76s 4.82s
Emit Time 6.14s (± 1.76%) 6.12s (± 2.46%) -0.02s (- 0.41%) 5.83s 6.55s
Total Time 13.97s (± 0.83%) 13.95s (± 1.03%) -0.02s (- 0.13%) 13.63s 14.34s
Monaco - node (v8.9.0, x86)
Memory used 204,190k (± 0.01%) 204,176k (± 0.02%) -15k (- 0.01%) 204,094k 204,270k
Parse Time 1.61s (± 0.51%) 1.61s (± 0.59%) +0.00s (+ 0.06%) 1.59s 1.64s
Bind Time 0.75s (± 1.56%) 0.74s (± 1.42%) -0.01s (- 0.80%) 0.73s 0.78s
Check Time 5.34s (± 1.13%) 5.35s (± 0.75%) +0.01s (+ 0.13%) 5.20s 5.40s
Emit Time 2.89s (± 2.16%) 2.88s (± 2.27%) -0.01s (- 0.35%) 2.82s 3.13s
Total Time 10.59s (± 0.43%) 10.58s (± 0.33%) -0.01s (- 0.08%) 10.52s 10.68s
TFS - node (v8.9.0, x86)
Memory used 180,903k (± 0.02%) 180,926k (± 0.02%) +22k (+ 0.01%) 180,803k 180,982k
Parse Time 1.30s (± 0.57%) 1.32s (± 1.09%) +0.01s (+ 1.15%) 1.29s 1.36s
Bind Time 0.69s (± 0.94%) 0.69s (± 1.09%) +0.00s (+ 0.29%) 0.68s 0.71s
Check Time 4.47s (± 0.65%) 4.50s (± 0.74%) +0.03s (+ 0.65%) 4.43s 4.57s
Emit Time 2.97s (± 0.92%) 2.96s (± 0.32%) -0.01s (- 0.37%) 2.94s 2.98s
Total Time 9.44s (± 0.43%) 9.47s (± 0.48%) +0.04s (+ 0.38%) 9.37s 9.59s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-161-generic
Architecturex64
Available Memory16 GB
Available Memory6 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
Benchmark Name Iterations
Current 34521 10
Baseline master 10

@amcasey amcasey added the Domain: Performance Reports of unusually slow behavior label Oct 16, 2019
@DanielRosenwasser
Copy link
Member

DanielRosenwasser commented Oct 17, 2019

👍 but get sign off from others.

fileName,
data,
writeByteOrderMark,
(f, d, w) => writeFileWorker(f, d, w),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameter names seem not very friendly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to convey that they're just threaded through to writeFileWorker. If full names are more conventional, I'm happy to expand them.

@amcasey amcasey merged commit 8cf1324 into microsoft:master Oct 22, 2019
@amcasey amcasey deleted the LazyDirectoryCreation branch October 22, 2019 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Domain: Performance Reports of unusually slow behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants