Don't cache Ternary.Maybe results when recursion is encountered during variance measurement

[ahejlsberg]

Fixes #39947.

[ahejlsberg]

@typescript-bot test this
@typescript-bot user test this
@typescript-bot run dt
@typescript-bot perf test this

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at 39724bd. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the perf test suite on this PR at 39724bd. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the extended test suite on this PR at 39724bd. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized community code test suite on this PR at 39724bd. You can monitor the build here.

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric	master	41218	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	349,757k (± 0.02%)	350,671k (± 0.02%)	+914k (+ 0.26%)	350,530k	350,785k
Parse Time	2.01s (± 0.80%)	2.02s (± 0.45%)	+0.01s (+ 0.50%)	2.00s	2.04s
Bind Time	0.82s (± 1.07%)	0.83s (± 1.13%)	+0.01s (+ 1.22%)	0.82s	0.86s
Check Time	4.94s (± 0.54%)	5.42s (± 0.33%)	+0.48s (+ 9.82%)	5.38s	5.47s
Emit Time	5.19s (± 0.41%)	5.25s (± 0.54%)	+0.06s (+ 1.19%)	5.20s	5.33s
Total Time	12.96s (± 0.34%)	13.53s (± 0.27%)	+0.57s (+ 4.37%)	13.45s	13.60s
Monaco - node (v10.16.3, x64)
Memory used	354,351k (± 0.02%)	354,544k (± 0.02%)	+192k (+ 0.05%)	354,337k	354,707k
Parse Time	1.57s (± 0.57%)	1.57s (± 0.43%)	+0.01s (+ 0.51%)	1.56s	1.59s
Bind Time	0.71s (± 0.56%)	0.72s (± 1.14%)	+0.01s (+ 0.84%)	0.70s	0.74s
Check Time	5.07s (± 0.54%)	5.53s (± 0.75%)	+0.46s (+ 9.13%)	5.44s	5.63s
Emit Time	2.76s (± 0.93%)	2.79s (± 0.72%)	+0.02s (+ 0.80%)	2.74s	2.84s
Total Time	10.11s (± 0.37%)	10.61s (± 0.51%)	+0.50s (+ 4.91%)	10.50s	10.77s
TFS - node (v10.16.3, x64)
Memory used	307,640k (± 0.04%)	307,844k (± 0.02%)	+204k (+ 0.07%)	307,636k	308,004k
Parse Time	1.22s (± 0.76%)	1.23s (± 0.45%)	+0.01s (+ 0.49%)	1.22s	1.24s
Bind Time	0.66s (± 1.37%)	0.67s (± 0.78%)	+0.01s (+ 1.21%)	0.65s	0.68s
Check Time	4.57s (± 0.94%)	5.04s (± 0.76%)	+0.47s (+10.33%)	4.95s	5.13s
Emit Time	2.90s (± 1.72%)	2.87s (± 0.94%)	-0.03s (- 0.97%)	2.81s	2.92s
Total Time	9.35s (± 0.59%)	9.81s (± 0.36%)	+0.46s (+ 4.93%)	9.73s	9.87s
material-ui - node (v10.16.3, x64)
Memory used	489,125k (± 0.02%)	489,213k (± 0.01%)	+88k (+ 0.02%)	489,110k	489,335k
Parse Time	1.99s (± 0.54%)	2.00s (± 0.65%)	+0.01s (+ 0.30%)	1.97s	2.03s
Bind Time	0.65s (± 0.90%)	0.65s (± 1.02%)	+0.00s (+ 0.46%)	0.63s	0.66s
Check Time	13.42s (± 0.51%)	13.83s (± 0.58%)	+0.41s (+ 3.02%)	13.68s	14.10s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	16.06s (± 0.45%)	16.48s (± 0.49%)	+0.42s (+ 2.60%)	16.34s	16.73s
Angular - node (v12.1.0, x64)
Memory used	326,934k (± 0.02%)	327,583k (± 0.10%)	+649k (+ 0.20%)	326,547k	327,909k
Parse Time	2.00s (± 0.53%)	2.02s (± 0.76%)	+0.02s (+ 0.80%)	1.98s	2.05s
Bind Time	0.81s (± 1.12%)	0.82s (± 2.45%)	+0.01s (+ 1.61%)	0.80s	0.89s
Check Time	4.83s (± 0.40%)	5.37s (± 0.67%)	+0.53s (+10.98%)	5.29s	5.45s
Emit Time	5.39s (± 0.84%)	5.43s (± 1.09%)	+0.03s (+ 0.63%)	5.34s	5.54s
Total Time	13.04s (± 0.52%)	13.63s (± 0.69%)	+0.59s (+ 4.56%)	13.44s	13.84s
Monaco - node (v12.1.0, x64)
Memory used	336,578k (± 0.02%)	336,780k (± 0.01%)	+202k (+ 0.06%)	336,735k	336,858k
Parse Time	1.54s (± 0.72%)	1.55s (± 0.61%)	+0.02s (+ 1.11%)	1.53s	1.58s
Bind Time	0.69s (± 0.58%)	0.70s (± 1.21%)	+0.01s (+ 1.45%)	0.69s	0.73s
Check Time	4.88s (± 0.50%)	5.38s (± 0.47%)	+0.50s (+10.26%)	5.33s	5.44s
Emit Time	2.82s (± 0.91%)	2.82s (± 0.75%)	+0.00s (+ 0.04%)	2.79s	2.88s
Total Time	9.93s (± 0.39%)	10.46s (± 0.40%)	+0.53s (+ 5.32%)	10.41s	10.57s
TFS - node (v12.1.0, x64)
Memory used	291,933k (± 0.03%)	292,039k (± 0.02%)	+105k (+ 0.04%)	291,905k	292,196k
Parse Time	1.23s (± 0.49%)	1.25s (± 0.85%)	+0.02s (+ 1.71%)	1.22s	1.27s
Bind Time	0.65s (± 1.38%)	0.64s (± 0.63%)	-0.01s (- 1.08%)	0.63s	0.65s
Check Time	4.47s (± 0.30%)	4.98s (± 0.60%)	+0.51s (+11.40%)	4.92s	5.05s
Emit Time	2.92s (± 0.99%)	2.95s (± 0.75%)	+0.03s (+ 0.99%)	2.92s	3.02s
Total Time	9.26s (± 0.36%)	9.81s (± 0.39%)	+0.55s (+ 6.00%)	9.76s	9.92s
material-ui - node (v12.1.0, x64)
Memory used	467,034k (± 0.05%)	467,253k (± 0.05%)	+219k (+ 0.05%)	466,303k	467,488k
Parse Time	2.01s (± 0.37%)	2.04s (± 0.52%)	+0.02s (+ 1.14%)	2.02s	2.06s
Bind Time	0.64s (± 1.05%)	0.64s (± 0.78%)	-0.00s (- 0.31%)	0.63s	0.65s
Check Time	12.05s (± 0.94%)	12.48s (± 0.86%)	+0.43s (+ 3.56%)	12.28s	12.74s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	14.70s (± 0.81%)	15.15s (± 0.76%)	+0.45s (+ 3.05%)	14.95s	15.44s
Angular - node (v8.9.0, x64)
Memory used	346,442k (± 0.01%)	347,337k (± 0.02%)	+895k (+ 0.26%)	347,185k	347,428k
Parse Time	2.54s (± 0.51%)	2.57s (± 0.41%)	+0.02s (+ 0.79%)	2.54s	2.59s
Bind Time	0.85s (± 0.43%)	0.87s (± 0.81%)	+0.02s (+ 1.76%)	0.85s	0.88s
Check Time	5.56s (± 0.52%)	5.98s (± 0.34%)	+0.42s (+ 7.52%)	5.93s	6.01s
Emit Time	6.15s (± 1.07%)	6.21s (± 1.47%)	+0.06s (+ 1.06%)	6.07s	6.46s
Total Time	15.11s (± 0.55%)	15.62s (± 0.71%)	+0.51s (+ 3.40%)	15.45s	15.90s
Monaco - node (v8.9.0, x64)
Memory used	355,704k (± 0.02%)	355,800k (± 0.01%)	+95k (+ 0.03%)	355,708k	355,884k
Parse Time	1.89s (± 0.77%)	1.89s (± 0.44%)	+0.01s (+ 0.42%)	1.88s	1.92s
Bind Time	0.89s (± 0.45%)	0.90s (± 0.53%)	+0.01s (+ 0.67%)	0.89s	0.91s
Check Time	5.62s (± 0.37%)	6.13s (± 1.11%)	+0.51s (+ 9.00%)	6.04s	6.36s
Emit Time	3.29s (± 1.17%)	3.26s (± 2.86%)	-0.02s (- 0.67%)	2.89s	3.34s
Total Time	11.69s (± 0.28%)	12.19s (± 0.48%)	+0.50s (+ 4.26%)	12.02s	12.30s
TFS - node (v8.9.0, x64)
Memory used	309,391k (± 0.02%)	309,499k (± 0.02%)	+108k (+ 0.03%)	309,396k	309,657k
Parse Time	1.55s (± 0.64%)	1.55s (± 0.38%)	+0.00s (+ 0.06%)	1.54s	1.57s
Bind Time	0.68s (± 0.73%)	0.68s (± 0.59%)	+0.00s (+ 0.44%)	0.67s	0.69s
Check Time	5.31s (± 0.58%)	5.77s (± 0.42%)	+0.46s (+ 8.61%)	5.73s	5.84s
Emit Time	2.94s (± 0.34%)	2.98s (± 1.00%)	+0.04s (+ 1.22%)	2.89s	3.02s
Total Time	10.48s (± 0.34%)	10.98s (± 0.26%)	+0.49s (+ 4.71%)	10.92s	11.05s
material-ui - node (v8.9.0, x64)
Memory used	493,431k (± 0.01%)	493,520k (± 0.01%)	+89k (+ 0.02%)	493,350k	493,659k
Parse Time	2.40s (± 0.46%)	2.42s (± 0.57%)	+0.02s (+ 0.67%)	2.40s	2.47s
Bind Time	0.81s (± 1.02%)	0.81s (± 1.36%)	-0.00s (- 0.49%)	0.79s	0.84s
Check Time	17.95s (± 0.69%)	18.34s (± 0.75%)	+0.39s (+ 2.16%)	18.06s	18.59s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	21.17s (± 0.56%)	21.57s (± 0.70%)	+0.40s (+ 1.88%)	21.27s	21.87s
Angular - node (v8.9.0, x86)
Memory used	198,690k (± 0.01%)	199,201k (± 0.03%)	+511k (+ 0.26%)	199,106k	199,331k
Parse Time	2.48s (± 1.02%)	2.50s (± 0.94%)	+0.02s (+ 0.89%)	2.46s	2.55s
Bind Time	1.00s (± 1.00%)	1.02s (± 1.46%)	+0.02s (+ 1.69%)	0.98s	1.04s
Check Time	5.05s (± 0.56%)	5.50s (± 0.76%)	+0.45s (+ 8.91%)	5.40s	5.57s
Emit Time	5.89s (± 0.83%)	6.02s (± 1.04%)	+0.14s (+ 2.29%)	5.94s	6.20s
Total Time	14.42s (± 0.58%)	15.03s (± 0.60%)	+0.62s (+ 4.28%)	14.82s	15.27s
Monaco - node (v8.9.0, x86)
Memory used	201,493k (± 0.01%)	201,591k (± 0.02%)	+98k (+ 0.05%)	201,467k	201,665k
Parse Time	1.94s (± 1.05%)	1.95s (± 0.91%)	+0.00s (+ 0.15%)	1.91s	2.00s
Bind Time	0.71s (± 0.47%)	0.71s (± 0.71%)	-0.00s (- 0.70%)	0.70s	0.72s
Check Time	5.47s (± 0.38%)	6.18s (± 0.51%)	+0.71s (+12.97%)	6.12s	6.25s
Emit Time	3.06s (± 0.59%)	2.71s (± 0.78%)	🟩-0.35s (-11.35%)	2.67s	2.77s
Total Time	11.18s (± 0.25%)	11.54s (± 0.28%)	+0.36s (+ 3.22%)	11.47s	11.60s
TFS - node (v8.9.0, x86)
Memory used	176,867k (± 0.03%)	176,963k (± 0.02%)	+96k (+ 0.05%)	176,903k	177,022k
Parse Time	1.60s (± 1.10%)	1.60s (± 1.29%)	-0.00s (- 0.25%)	1.57s	1.66s
Bind Time	0.66s (± 2.04%)	0.64s (± 0.76%)	-0.01s (- 1.68%)	0.64s	0.66s
Check Time	4.82s (± 0.58%)	5.32s (± 0.67%)	+0.50s (+10.33%)	5.24s	5.39s
Emit Time	2.80s (± 0.66%)	2.85s (± 1.08%)	+0.04s (+ 1.60%)	2.80s	2.92s
Total Time	9.88s (± 0.51%)	10.41s (± 0.53%)	+0.53s (+ 5.33%)	10.28s	10.55s
material-ui - node (v8.9.0, x86)
Memory used	277,878k (± 0.02%)	277,958k (± 0.02%)	+81k (+ 0.03%)	277,801k	278,089k
Parse Time	2.46s (± 0.43%)	2.48s (± 0.62%)	+0.02s (+ 1.02%)	2.45s	2.51s
Bind Time	0.69s (± 1.35%)	0.70s (± 2.22%)	+0.02s (+ 2.47%)	0.67s	0.75s
Check Time	16.45s (± 0.73%)	16.83s (± 0.81%)	+0.38s (+ 2.30%)	16.54s	17.20s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	19.60s (± 0.63%)	20.02s (± 0.79%)	+0.42s (+ 2.14%)	19.66s	20.40s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	2 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	41218	10
Baseline	master	10

[typescript-bot]

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

[weswigham]

recursiveVariances should be un-set by recursiveTypeRelatedTo once we want to cache a non-maybe value, no? So if there is, thereafter, a Maybe during the same comparison that is not variance related, it is cache able?

[ahejlsberg]

@weswigham In some sense we really want to have two different Ternary.Maybe values: One that originates in the depth limiter (the original one) and one that originates in a recursive variance measurement (the new one). The new one would be "infectious" and we would never put it in the cache. What's implemented now is sort of a blunt approximation of that, but once we have a Ternary.Maybe that originated in recursive variance measurement we have to assume every Ternary.Maybe is possibly one of those and thus not cacheable.

[ahejlsberg]

Hmm, the effect on performance is definitely unacceptable. It might be better with the two kinds of Ternary.Maybe I describe above, but I'm not sure. Unfortunately it's not a small amount of work to find out.

[ahejlsberg]

Actually, wasn't too hard to introduce a Ternary.Unknown that acts like a signaling Ternary.Maybe. We'll see if that helps performance.

[ahejlsberg]

@typescript-bot perf test this

[typescript-bot]

Heya @ahejlsberg, I've started to run the perf test suite on this PR at dcfce3f. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric	master	41218	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	2 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	41218	10
Baseline	master	10

[ahejlsberg]

@rbuckton Hmm, the reporting from that last test runs seems broken. Any ideas?

[ahejlsberg]

@typescript-bot perf test this

[typescript-bot]

Heya @ahejlsberg, I've started to run the perf test suite on this PR at dcfce3f. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric	master	41218	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v10.16.3, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v12.1.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v8.9.0, x64)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Angular - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Monaco - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
TFS - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
material-ui - node (v8.9.0, x86)
Memory used	0k	0k	0k ( NaN%)	0k	0k
Parse Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Bind Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Check Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Emit Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s
Total Time	0.00s	0.00s	0.00s ( NaN%)	0.00s	0.00s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	2 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	41218	10
Baseline	master	10

[rbuckton]

Might be related to the switch to the native performance API. I'll look into it.

[rbuckton]

#41253 will fix the issue with tracking performance data.

[rbuckton]

#41253 has been merged. If you merge the latest from master the perf results should be working now.

[ahejlsberg]

@typescript-bot perf test this

[typescript-bot]

Heya @ahejlsberg, I've started to run the perf test suite on this PR at 2b8ca38. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric	master	41218	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	350,683k (± 0.02%)	350,670k (± 0.02%)	-13k (- 0.00%)	350,553k	350,978k
Parse Time	2.09s (± 0.38%)	2.09s (± 0.28%)	-0.00s (- 0.10%)	2.08s	2.10s
Bind Time	0.84s (± 1.03%)	0.84s (± 0.35%)	+0.00s (+ 0.48%)	0.84s	0.85s
Check Time	5.44s (± 0.41%)	5.43s (± 0.37%)	-0.01s (- 0.13%)	5.39s	5.47s
Emit Time	5.34s (± 0.40%)	5.36s (± 1.20%)	+0.03s (+ 0.47%)	5.27s	5.54s
Total Time	13.71s (± 0.27%)	13.72s (± 0.50%)	+0.02s (+ 0.12%)	13.62s	13.90s
Monaco - node (v10.16.3, x64)
Memory used	354,753k (± 0.02%)	354,735k (± 0.02%)	-18k (- 0.01%)	354,559k	354,909k
Parse Time	1.62s (± 0.21%)	1.61s (± 0.60%)	-0.01s (- 0.37%)	1.59s	1.64s
Bind Time	0.72s (± 0.72%)	0.71s (± 0.83%)	-0.01s (- 0.70%)	0.70s	0.72s
Check Time	5.56s (± 0.48%)	5.56s (± 0.50%)	-0.00s (- 0.02%)	5.49s	5.65s
Emit Time	2.81s (± 0.71%)	2.80s (± 0.56%)	-0.00s (- 0.18%)	2.77s	2.84s
Total Time	10.70s (± 0.42%)	10.69s (± 0.32%)	-0.01s (- 0.13%)	10.62s	10.78s
TFS - node (v10.16.3, x64)
Memory used	307,908k (± 0.02%)	307,983k (± 0.03%)	+75k (+ 0.02%)	307,816k	308,252k
Parse Time	1.24s (± 0.59%)	1.24s (± 0.42%)	-0.00s (- 0.24%)	1.23s	1.25s
Bind Time	0.68s (± 1.41%)	0.67s (± 0.99%)	-0.00s (- 0.30%)	0.66s	0.69s
Check Time	5.05s (± 0.69%)	5.05s (± 0.55%)	+0.00s (+ 0.02%)	5.00s	5.12s
Emit Time	2.94s (± 1.14%)	2.88s (± 1.08%)	-0.06s (- 2.11%)	2.80s	2.95s
Total Time	9.91s (± 0.53%)	9.84s (± 0.32%)	-0.07s (- 0.68%)	9.75s	9.90s
material-ui - node (v10.16.3, x64)
Memory used	489,304k (± 0.01%)	489,314k (± 0.01%)	+10k (+ 0.00%)	489,188k	489,421k
Parse Time	2.08s (± 0.34%)	2.08s (± 0.50%)	-0.00s (- 0.05%)	2.06s	2.11s
Bind Time	0.65s (± 0.85%)	0.65s (± 0.69%)	-0.00s (- 0.61%)	0.64s	0.66s
Check Time	13.98s (± 0.70%)	13.93s (± 0.87%)	-0.06s (- 0.40%)	13.75s	14.36s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	16.72s (± 0.60%)	16.66s (± 0.75%)	-0.06s (- 0.38%)	16.46s	17.09s
Angular - node (v12.1.0, x64)
Memory used	327,887k (± 0.02%)	327,896k (± 0.02%)	+9k (+ 0.00%)	327,711k	328,079k
Parse Time	2.09s (± 0.57%)	2.09s (± 0.42%)	-0.01s (- 0.29%)	2.07s	2.11s
Bind Time	0.82s (± 0.94%)	0.82s (± 0.73%)	-0.00s (- 0.61%)	0.81s	0.83s
Check Time	5.40s (± 0.67%)	5.39s (± 0.67%)	-0.01s (- 0.19%)	5.30s	5.47s
Emit Time	5.49s (± 0.62%)	5.50s (± 0.60%)	+0.00s (+ 0.07%)	5.42s	5.59s
Total Time	13.81s (± 0.45%)	13.79s (± 0.39%)	-0.02s (- 0.12%)	13.62s	13.92s
Monaco - node (v12.1.0, x64)
Memory used	336,815k (± 0.01%)	336,832k (± 0.01%)	+18k (+ 0.01%)	336,696k	336,919k
Parse Time	1.60s (± 0.62%)	1.60s (± 0.96%)	-0.00s (- 0.06%)	1.57s	1.65s
Bind Time	0.70s (± 0.96%)	0.70s (± 0.52%)	-0.00s (- 0.14%)	0.69s	0.70s
Check Time	5.41s (± 0.49%)	5.42s (± 0.71%)	+0.01s (+ 0.26%)	5.37s	5.54s
Emit Time	2.89s (± 1.35%)	2.87s (± 1.13%)	-0.02s (- 0.52%)	2.81s	2.96s
Total Time	10.60s (± 0.49%)	10.59s (± 0.60%)	-0.01s (- 0.07%)	10.48s	10.79s
TFS - node (v12.1.0, x64)
Memory used	292,242k (± 0.02%)	292,212k (± 0.03%)	-30k (- 0.01%)	292,044k	292,376k
Parse Time	1.26s (± 0.82%)	1.26s (± 0.85%)	-0.00s (- 0.08%)	1.24s	1.29s
Bind Time	0.65s (± 1.08%)	0.65s (± 1.36%)	-0.00s (- 0.61%)	0.63s	0.67s
Check Time	5.04s (± 0.68%)	5.02s (± 0.22%)	-0.02s (- 0.40%)	4.99s	5.04s
Emit Time	2.97s (± 1.14%)	2.95s (± 0.68%)	-0.01s (- 0.44%)	2.91s	3.00s
Total Time	9.92s (± 0.60%)	9.88s (± 0.33%)	-0.04s (- 0.39%)	9.82s	9.97s
material-ui - node (v12.1.0, x64)
Memory used	467,310k (± 0.05%)	466,926k (± 0.10%)	-384k (- 0.08%)	466,025k	467,588k
Parse Time	2.10s (± 0.63%)	2.08s (± 0.69%)	-0.02s (- 0.95%)	2.05s	2.11s
Bind Time	0.64s (± 0.90%)	0.64s (± 1.18%)	+0.00s (+ 0.16%)	0.63s	0.67s
Check Time	12.56s (± 1.19%)	12.50s (± 0.84%)	-0.06s (- 0.49%)	12.32s	12.76s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.31s (± 1.00%)	15.23s (± 0.70%)	-0.08s (- 0.49%)	15.04s	15.49s
Angular - node (v8.9.0, x64)
Memory used	353,106k (± 0.02%)	353,068k (± 0.01%)	-38k (- 0.01%)	352,916k	353,156k
Parse Time	2.65s (± 0.31%)	2.66s (± 0.26%)	+0.00s (+ 0.19%)	2.65s	2.67s
Bind Time	0.88s (± 1.25%)	0.88s (± 0.77%)	+0.00s (+ 0.11%)	0.87s	0.90s
Check Time	6.05s (± 0.45%)	6.00s (± 0.37%)	-0.05s (- 0.76%)	5.96s	6.05s
Emit Time	6.48s (± 0.82%)	6.35s (± 1.25%)	-0.13s (- 1.99%)	6.16s	6.49s
Total Time	16.07s (± 0.48%)	15.90s (± 0.48%)	-0.17s (- 1.05%)	15.70s	16.02s
Monaco - node (v8.9.0, x64)
Memory used	358,497k (± 0.01%)	358,537k (± 0.02%)	+40k (+ 0.01%)	358,372k	358,645k
Parse Time	1.94s (± 0.30%)	1.95s (± 0.52%)	+0.01s (+ 0.31%)	1.93s	1.98s
Bind Time	0.90s (± 0.54%)	0.90s (± 0.86%)	-0.01s (- 0.55%)	0.89s	0.92s
Check Time	6.17s (± 0.94%)	6.14s (± 0.69%)	-0.03s (- 0.53%)	6.04s	6.25s
Emit Time	3.29s (± 2.46%)	3.33s (± 0.51%)	+0.04s (+ 1.12%)	3.30s	3.38s
Total Time	12.31s (± 0.43%)	12.32s (± 0.37%)	+0.00s (+ 0.03%)	12.21s	12.41s
TFS - node (v8.9.0, x64)
Memory used	310,333k (± 0.02%)	310,333k (± 0.01%)	+0k (+ 0.00%)	310,224k	310,420k
Parse Time	1.58s (± 0.52%)	1.58s (± 0.35%)	-0.00s (- 0.13%)	1.57s	1.59s
Bind Time	0.68s (± 0.33%)	0.68s (± 0.73%)	-0.00s (- 0.73%)	0.66s	0.68s
Check Time	5.80s (± 0.56%)	5.76s (± 0.62%)	-0.04s (- 0.66%)	5.71s	5.88s
Emit Time	2.99s (± 0.91%)	2.96s (± 0.97%)	-0.03s (- 1.07%)	2.89s	2.99s
Total Time	11.05s (± 0.47%)	10.98s (± 0.30%)	-0.07s (- 0.66%)	10.87s	11.04s
material-ui - node (v8.9.0, x64)
Memory used	496,234k (± 0.01%)	496,241k (± 0.01%)	+8k (+ 0.00%)	496,088k	496,472k
Parse Time	2.52s (± 0.53%)	2.52s (± 0.46%)	-0.00s (- 0.12%)	2.49s	2.54s
Bind Time	0.82s (± 1.90%)	0.82s (± 1.33%)	-0.00s (- 0.61%)	0.79s	0.84s
Check Time	18.44s (± 1.31%)	18.41s (± 0.71%)	-0.03s (- 0.17%)	18.16s	18.65s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	21.79s (± 1.18%)	21.74s (± 0.61%)	-0.04s (- 0.19%)	21.50s	22.01s
Angular - node (v8.9.0, x86)
Memory used	202,109k (± 0.02%)	202,104k (± 0.03%)	-6k (- 0.00%)	201,990k	202,239k
Parse Time	2.57s (± 0.68%)	2.57s (± 1.10%)	-0.00s (- 0.04%)	2.54s	2.65s
Bind Time	1.01s (± 1.01%)	1.00s (± 0.91%)	-0.01s (- 0.99%)	0.99s	1.03s
Check Time	5.52s (± 0.44%)	5.52s (± 0.77%)	0.00s ( 0.00%)	5.43s	5.63s
Emit Time	6.06s (± 0.99%)	6.07s (± 0.79%)	+0.00s (+ 0.03%)	5.95s	6.21s
Total Time	15.17s (± 0.50%)	15.16s (± 0.41%)	-0.00s (- 0.02%)	15.03s	15.28s
Monaco - node (v8.9.0, x86)
Memory used	202,954k (± 0.02%)	202,947k (± 0.01%)	-7k (- 0.00%)	202,895k	202,981k
Parse Time	1.98s (± 0.71%)	1.98s (± 1.03%)	-0.00s (- 0.10%)	1.96s	2.06s
Bind Time	0.71s (± 0.78%)	0.71s (± 0.73%)	-0.00s (- 0.14%)	0.70s	0.72s
Check Time	6.22s (± 0.64%)	6.22s (± 0.88%)	+0.00s (+ 0.02%)	6.15s	6.42s
Emit Time	2.76s (± 0.69%)	2.77s (± 1.01%)	+0.01s (+ 0.54%)	2.73s	2.86s
Total Time	11.67s (± 0.48%)	11.69s (± 0.57%)	+0.02s (+ 0.14%)	11.56s	11.87s
TFS - node (v8.9.0, x86)
Memory used	177,392k (± 0.04%)	177,415k (± 0.01%)	+22k (+ 0.01%)	177,375k	177,475k
Parse Time	1.63s (± 0.89%)	1.63s (± 0.77%)	0.00s ( 0.00%)	1.60s	1.66s
Bind Time	0.64s (± 0.90%)	0.64s (± 0.46%)	-0.00s (- 0.16%)	0.64s	0.65s
Check Time	5.34s (± 1.05%)	5.36s (± 0.90%)	+0.02s (+ 0.34%)	5.28s	5.50s
Emit Time	2.83s (± 1.50%)	2.83s (± 1.01%)	-0.00s (- 0.18%)	2.76s	2.89s
Total Time	10.45s (± 0.90%)	10.46s (± 0.46%)	+0.01s (+ 0.12%)	10.36s	10.59s
material-ui - node (v8.9.0, x86)
Memory used	279,337k (± 0.01%)	279,319k (± 0.01%)	-18k (- 0.01%)	279,239k	279,400k
Parse Time	2.57s (± 0.72%)	2.57s (± 0.47%)	-0.00s (- 0.12%)	2.54s	2.60s
Bind Time	0.77s (± 7.52%)	0.76s (± 5.57%)	-0.01s (- 1.69%)	0.69s	0.87s
Check Time	16.90s (± 0.88%)	16.90s (± 0.86%)	+0.01s (+ 0.05%)	16.63s	17.35s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	20.24s (± 0.61%)	20.23s (± 0.73%)	-0.01s (- 0.04%)	19.99s	20.63s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	2 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	41218	10
Baseline	master	10

[ahejlsberg]

Performance looks good now and issues in test suites all appear to be preexisting conditions.

[ahejlsberg]

@typescript-bot perf test this

[typescript-bot]

Heya @ahejlsberg, I've started to run the perf test suite on this PR at a132c04. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric	master	41218	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	350,615k (± 0.02%)	350,588k (± 0.02%)	-27k (- 0.01%)	350,398k	350,715k
Parse Time	2.07s (± 0.70%)	2.09s (± 0.61%)	+0.02s (+ 0.82%)	2.06s	2.12s
Bind Time	0.84s (± 0.66%)	0.84s (± 0.71%)	+0.00s (+ 0.36%)	0.83s	0.86s
Check Time	4.96s (± 0.46%)	4.97s (± 0.51%)	+0.01s (+ 0.24%)	4.92s	5.04s
Emit Time	5.31s (± 0.69%)	5.30s (± 0.38%)	-0.01s (- 0.13%)	5.26s	5.34s
Total Time	13.19s (± 0.41%)	13.21s (± 0.36%)	+0.02s (+ 0.17%)	13.11s	13.33s
Monaco - node (v10.16.3, x64)
Memory used	354,587k (± 0.01%)	354,589k (± 0.01%)	+1k (+ 0.00%)	354,453k	354,685k
Parse Time	1.60s (± 0.66%)	1.60s (± 0.52%)	+0.00s (+ 0.12%)	1.59s	1.62s
Bind Time	0.72s (± 0.56%)	0.72s (± 0.77%)	-0.00s (- 0.14%)	0.71s	0.73s
Check Time	5.13s (± 0.35%)	5.11s (± 0.56%)	-0.01s (- 0.29%)	5.07s	5.20s
Emit Time	2.79s (± 0.76%)	2.80s (± 0.67%)	+0.01s (+ 0.39%)	2.76s	2.83s
Total Time	10.23s (± 0.29%)	10.23s (± 0.45%)	-0.00s (- 0.01%)	10.16s	10.37s
TFS - node (v10.16.3, x64)
Memory used	307,837k (± 0.02%)	307,823k (± 0.02%)	-14k (- 0.00%)	307,716k	307,996k
Parse Time	1.23s (± 0.57%)	1.24s (± 0.80%)	+0.01s (+ 0.57%)	1.22s	1.26s
Bind Time	0.67s (± 0.71%)	0.68s (± 0.50%)	+0.00s (+ 0.59%)	0.67s	0.68s
Check Time	4.56s (± 0.58%)	4.61s (± 0.80%)	+0.05s (+ 1.01%)	4.52s	4.68s
Emit Time	2.93s (± 0.70%)	2.94s (± 1.22%)	+0.01s (+ 0.27%)	2.86s	3.01s
Total Time	9.39s (± 0.47%)	9.46s (± 0.67%)	+0.07s (+ 0.72%)	9.34s	9.60s
material-ui - node (v10.16.3, x64)
Memory used	489,242k (± 0.01%)	489,283k (± 0.01%)	+41k (+ 0.01%)	489,184k	489,407k
Parse Time	2.07s (± 0.56%)	2.08s (± 0.53%)	+0.01s (+ 0.58%)	2.05s	2.09s
Bind Time	0.65s (± 0.89%)	0.65s (± 0.61%)	-0.00s (- 0.31%)	0.64s	0.66s
Check Time	13.56s (± 0.80%)	13.60s (± 0.74%)	+0.05s (+ 0.37%)	13.43s	13.84s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	16.27s (± 0.72%)	16.34s (± 0.66%)	+0.06s (+ 0.39%)	16.15s	16.58s
Angular - node (v12.1.0, x64)
Memory used	327,792k (± 0.02%)	327,685k (± 0.09%)	-106k (- 0.03%)	326,596k	328,013k
Parse Time	2.07s (± 0.64%)	2.08s (± 0.78%)	+0.01s (+ 0.39%)	2.05s	2.12s
Bind Time	0.81s (± 1.02%)	0.82s (± 1.69%)	+0.01s (+ 0.61%)	0.80s	0.87s
Check Time	4.92s (± 0.47%)	4.91s (± 0.38%)	-0.01s (- 0.12%)	4.87s	4.97s
Emit Time	5.50s (± 0.57%)	5.56s (± 1.21%)	+0.05s (+ 0.93%)	5.44s	5.77s
Total Time	13.31s (± 0.20%)	13.37s (± 0.63%)	+0.06s (+ 0.42%)	13.21s	13.64s
Monaco - node (v12.1.0, x64)
Memory used	336,733k (± 0.02%)	336,719k (± 0.01%)	-14k (- 0.00%)	336,627k	336,803k
Parse Time	1.58s (± 0.64%)	1.58s (± 0.63%)	-0.00s (- 0.06%)	1.56s	1.61s
Bind Time	0.70s (± 0.49%)	0.70s (± 0.57%)	+0.00s (+ 0.57%)	0.69s	0.71s
Check Time	4.90s (± 0.29%)	4.93s (± 0.32%)	+0.03s (+ 0.55%)	4.88s	4.96s
Emit Time	2.87s (± 0.64%)	2.86s (± 0.86%)	-0.01s (- 0.17%)	2.82s	2.94s
Total Time	10.05s (± 0.27%)	10.07s (± 0.30%)	+0.02s (+ 0.20%)	9.99s	10.14s
TFS - node (v12.1.0, x64)
Memory used	292,070k (± 0.01%)	292,042k (± 0.02%)	-28k (- 0.01%)	291,927k	292,125k
Parse Time	1.25s (± 0.80%)	1.26s (± 0.84%)	+0.01s (+ 0.88%)	1.24s	1.28s
Bind Time	0.64s (± 1.04%)	0.65s (± 0.93%)	+0.00s (+ 0.31%)	0.63s	0.66s
Check Time	4.49s (± 0.44%)	4.49s (± 0.29%)	-0.00s (- 0.00%)	4.47s	4.52s
Emit Time	2.96s (± 0.49%)	2.97s (± 1.02%)	+0.00s (+ 0.03%)	2.87s	3.04s
Total Time	9.35s (± 0.23%)	9.37s (± 0.44%)	+0.02s (+ 0.18%)	9.28s	9.46s
material-ui - node (v12.1.0, x64)
Memory used	467,273k (± 0.05%)	467,363k (± 0.01%)	+89k (+ 0.02%)	467,167k	467,459k
Parse Time	2.07s (± 0.45%)	2.09s (± 0.60%)	+0.01s (+ 0.63%)	2.06s	2.11s
Bind Time	0.64s (± 0.87%)	0.64s (± 0.92%)	+0.00s (+ 0.31%)	0.63s	0.66s
Check Time	11.99s (± 0.49%)	12.06s (± 0.95%)	+0.07s (+ 0.58%)	11.88s	12.30s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	14.71s (± 0.41%)	14.79s (± 0.77%)	+0.09s (+ 0.58%)	14.58s	15.03s
Angular - node (v8.9.0, x64)
Memory used	352,969k (± 0.02%)	352,981k (± 0.01%)	+12k (+ 0.00%)	352,896k	353,149k
Parse Time	2.65s (± 0.47%)	2.65s (± 0.47%)	0.00s ( 0.00%)	2.63s	2.69s
Bind Time	0.88s (± 0.88%)	0.88s (± 0.63%)	-0.00s (- 0.11%)	0.87s	0.89s
Check Time	5.60s (± 0.55%)	5.62s (± 0.71%)	+0.02s (+ 0.41%)	5.51s	5.70s
Emit Time	6.28s (± 0.92%)	6.31s (± 1.16%)	+0.03s (+ 0.51%)	6.12s	6.43s
Total Time	15.41s (± 0.38%)	15.46s (± 0.53%)	+0.06s (+ 0.36%)	15.25s	15.63s
Monaco - node (v8.9.0, x64)
Memory used	358,422k (± 0.01%)	358,392k (± 0.01%)	-29k (- 0.01%)	358,276k	358,476k
Parse Time	1.94s (± 0.46%)	1.93s (± 0.53%)	-0.00s (- 0.26%)	1.92s	1.97s
Bind Time	0.90s (± 0.90%)	0.91s (± 0.40%)	+0.01s (+ 0.67%)	0.90s	0.91s
Check Time	5.63s (± 0.61%)	5.64s (± 0.64%)	+0.01s (+ 0.20%)	5.59s	5.75s
Emit Time	3.38s (± 0.90%)	3.40s (± 0.41%)	+0.02s (+ 0.74%)	3.37s	3.44s
Total Time	11.84s (± 0.39%)	11.88s (± 0.43%)	+0.04s (+ 0.35%)	11.79s	12.04s
TFS - node (v8.9.0, x64)
Memory used	310,267k (± 0.01%)	310,274k (± 0.01%)	+6k (+ 0.00%)	310,194k	310,360k
Parse Time	1.57s (± 0.51%)	1.58s (± 0.43%)	+0.01s (+ 0.51%)	1.56s	1.59s
Bind Time	0.68s (± 0.91%)	0.68s (± 0.70%)	+0.00s (+ 0.59%)	0.67s	0.69s
Check Time	5.30s (± 0.50%)	5.35s (± 0.73%)	+0.05s (+ 1.02%)	5.28s	5.44s
Emit Time	2.98s (± 0.98%)	2.99s (± 0.50%)	+0.01s (+ 0.23%)	2.96s	3.04s
Total Time	10.53s (± 0.47%)	10.60s (± 0.47%)	+0.07s (+ 0.67%)	10.53s	10.73s
material-ui - node (v8.9.0, x64)
Memory used	496,195k (± 0.02%)	496,258k (± 0.01%)	+63k (+ 0.01%)	496,134k	496,378k
Parse Time	2.50s (± 0.42%)	2.50s (± 0.66%)	+0.00s (+ 0.08%)	2.46s	2.55s
Bind Time	0.82s (± 1.43%)	0.82s (± 0.86%)	+0.00s (+ 0.49%)	0.80s	0.83s
Check Time	17.93s (± 0.62%)	18.15s (± 0.83%)	+0.22s (+ 1.24%)	17.71s	18.43s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	21.25s (± 0.56%)	21.47s (± 0.69%)	+0.23s (+ 1.06%)	21.05s	21.75s
Angular - node (v8.9.0, x86)
Memory used	202,008k (± 0.03%)	202,092k (± 0.03%)	+85k (+ 0.04%)	201,980k	202,215k
Parse Time	2.57s (± 0.84%)	2.56s (± 0.53%)	-0.00s (- 0.16%)	2.53s	2.59s
Bind Time	1.02s (± 1.36%)	1.02s (± 1.17%)	+0.00s (+ 0.30%)	1.00s	1.05s
Check Time	5.08s (± 0.46%)	5.08s (± 0.58%)	+0.00s (+ 0.02%)	5.02s	5.17s
Emit Time	6.09s (± 0.75%)	6.08s (± 0.99%)	-0.01s (- 0.15%)	5.94s	6.26s
Total Time	14.74s (± 0.39%)	14.74s (± 0.45%)	-0.01s (- 0.05%)	14.60s	14.88s
Monaco - node (v8.9.0, x86)
Memory used	202,929k (± 0.02%)	202,913k (± 0.02%)	-16k (- 0.01%)	202,811k	202,992k
Parse Time	1.99s (± 0.81%)	1.97s (± 0.52%)	-0.02s (- 0.90%)	1.95s	1.99s
Bind Time	0.72s (± 0.95%)	0.71s (± 0.69%)	-0.00s (- 0.28%)	0.70s	0.72s
Check Time	5.75s (± 0.74%)	5.74s (± 1.14%)	-0.01s (- 0.09%)	5.50s	5.85s
Emit Time	2.76s (± 1.50%)	2.80s (± 2.61%)	+0.04s (+ 1.56%)	2.70s	3.06s
Total Time	11.21s (± 0.40%)	11.23s (± 0.50%)	+0.02s (+ 0.19%)	11.15s	11.37s
TFS - node (v8.9.0, x86)
Memory used	177,394k (± 0.03%)	177,366k (± 0.02%)	-28k (- 0.02%)	177,284k	177,481k
Parse Time	1.62s (± 0.62%)	1.62s (± 1.32%)	+0.01s (+ 0.31%)	1.59s	1.69s
Bind Time	0.65s (± 0.72%)	0.65s (± 1.06%)	-0.01s (- 0.77%)	0.63s	0.66s
Check Time	4.86s (± 0.92%)	4.83s (± 0.58%)	-0.03s (- 0.72%)	4.77s	4.89s
Emit Time	2.84s (± 1.40%)	2.82s (± 0.92%)	-0.02s (- 0.78%)	2.75s	2.86s
Total Time	9.97s (± 0.83%)	9.91s (± 0.41%)	-0.06s (- 0.57%)	9.84s	10.00s
material-ui - node (v8.9.0, x86)
Memory used	279,273k (± 0.01%)	279,260k (± 0.01%)	-12k (- 0.00%)	279,179k	279,337k
Parse Time	2.55s (± 0.52%)	2.55s (± 0.57%)	+0.00s (+ 0.16%)	2.53s	2.60s
Bind Time	0.75s (± 6.64%)	0.73s (± 6.73%)	-0.02s (- 2.67%)	0.68s	0.86s
Check Time	16.41s (± 0.75%)	16.38s (± 0.65%)	-0.03s (- 0.18%)	16.16s	16.68s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	19.71s (± 0.44%)	19.67s (± 0.61%)	-0.04s (- 0.22%)	19.39s	19.94s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	1 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	41218	10
Baseline	master	10