TypeScript is crashed by using too much memory

[falsandtru]

This crash is caused by a few codes. Is this correct behavior?

TypeScript Version:

1.9.0-dev.20160609

Code

https://github.com/falsandtru/spica/tree/v0.0.6
https://travis-ci.org/falsandtru/spica/builds/136372602
https://github.com/falsandtru/spica/blob/v0.0.6/src/lib/monad/sequence.ts

$ $(npm bin)/tsc --noEmit -t es5 --diagnostics typings/tsd.d.ts typings/spica.d.ts typings/lib.ex.d.ts spica.ts

<--- Last few GCs --->

  168134 ms: Mark-sweep 1292.9 (1434.6) -> 1292.9 (1434.6) MB, 3959.0 / 0 ms [allocation failure] [GC in old space requested].
  172283 ms: Mark-sweep 1292.9 (1434.6) -> 1292.9 (1434.6) MB, 4149.0 / 0 ms [allocation failure] [GC in old space requested].
  176418 ms: Mark-sweep 1292.9 (1434.6) -> 1292.9 (1434.6) MB, 4135.0 / 0 ms [last resort gc].
  180433 ms: Mark-sweep 1292.9 (1434.6) -> 1292.9 (1434.6) MB, 4015.0 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000001F022BC9E79 <JS Object>
    1: objectTypeRelatedTo [*\spica\node_modules\typescript\lib\tsc.js:~17516] [pc=000002F3674B3CA8] (this=000001F022BE4519 <JS Global Object>,source=00000096942236C1 <a Type with map 0000038A0C559FA9>,originalSource=00000096942236C1 <a Type with map 0000038A0C559FA9>,target=000003ED5B64DCC9 <a Type with map 0000031C76BCBA19>,reportErrors=000001F022B04299 <false>)
...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

[mhegazy]

The source of the regression was #8713

[falsandtru]

v1.8.10 has a same problem, and sometimes v1.8.30 with VS2015 will be crashed a few minutes later. So I feel this problem has another reason different from #8713.

[mhegazy]

In 1.8.10, i see it finishing up compilation in less than 10 secs, but uses about 1 GB of memory. this commit is when it started to crash the compiler.

In general, these complex recursive definitions like the ones in https://github.com/falsandtru/spica/blob/v0.0.6/src/lib/monad/sequence.ts, do not fare well in a structural type system. the system needs to relate these generic types structurally, and every time it descends into these types, it instantiates their method signatures, which include other generic types, and so on.

This is similar to the issues we ran into with RxJS a while back, see #7344. the fix was to simplify the complexity of these types to avoid the expensive comparisons, see ReactiveX/rxjs#1475.

[falsandtru]

In short, crashes of VS won't be fixed? VS uses 2.5~3.0GB of memory.

[falsandtru]

I found a solution.

before:

$ $(npm bin)/tsc --noEmit -t es5 --diagnostics typings/tsd.d.ts typings/spica.d
.ts typings/lib.ex.d.ts spica.ts
Files:              34
Lines:           19533
Nodes:           94018
Identifiers:     35814
Symbols:       1964595
Types:          529786
Memory used:   828809K
I/O read:        0.03s
I/O write:       0.00s
Parse time:      0.88s
Bind time:       0.58s
Check time:     28.43s
Emit time:       0.00s
Total time:     29.89s

after:

- export class Sequence<T, S> extends Monad<T> implements ISequence<T, S> {
+ export class Sequence<T, S> extends Monad<T> {

$ $(npm bin)/tsc --noEmit -t es5 --diagnostics typings/tsd.d.ts typings/spica.d
.ts typings/lib.ex.d.ts spica.ts
Files:              34
Lines:           19533
Nodes:           94000
Identifiers:     35807
Symbols:        892012
Types:           41074
Memory used:   119040K
I/O read:        0.03s
I/O write:       0.00s
Parse time:      1.14s
Bind time:       0.62s
Check time:      5.84s
Emit time:       0.00s
Total time:      7.60s

Is this a correct behavior? It seems to leak Symbols and Types. I think, interfaces shouldn't impair the performance.

[mhegazy]

What do you mean by leak symbols?

[falsandtru]

The compiler uses too many symbols, types, memory and Check time only for implementation of interfaces.

- export class Sequence<T, S> extends Monad<T> implements ISequence<T, S> {
+ export class Sequence<T, S> extends Monad<T> {

- Symbols:       1964595
+ Symbols:        892012
- Types:          529786
+ Types:           41074
- Memory used:   828809K
+ Memory used:   119040K
- Check time:     28.43s
+ Check time:      5.84s

[sandersn]

Actually Sequence and ISequence are the same type -- sequence.ts imports Sequence as ISequence from spica, which exports everything from exports.ts, which imports Sequence from sequence.ts. The compiler should issue an error in this case.

Supervisor and ISupervisor have the same problem.

In my testing, spica v0.0.6 still crashes node even with these implements clauses removed.

[falsandtru]

I want to ensure type consistency of declaration(.d.ts) and implementation(.ts) such as class Sequence implements ISequence. How can I ensure this consistency after fixed?

[sandersn]

In the original code, Sequence was not checked against Sequence in spica.d.ts, but against itself. Actually, you don't need to ensure consistency of the implementation with the declaration. Just generate declarations with --declaration when compiling. Then you just need to ensure that the newly generated declaration is the same as the old version. Use source control for that.

For example, compile with --declaration. When you get ready to check in the code, make sure that the generated spica.d.ts has not changed. If it has changed, you'll see a diff and can decide whether the declaration changes are OK to check in.