Catastrophic performance decrease when adding overloads to a method (sometimes crashes tsc)

[Ptival]

TypeScript Version: 1.9.0

Code

interface IWhatever { }

interface IBug<T extends IWhatever> {
  foo$: Rx.Observable<{}>;
}

export class Bug<T extends IWhatever> implements IBug<T> {
  foo$: Rx.Subject<{}>;
}

Using:

    "rx": "registry:dt/rx#0.0.0+20160317120654",
    "rx-lite": "registry:dt/rx-lite#0.0.0+20160628114230",
    "rx.async-lite": "registry:dt/rx.async-lite#0.0.0+20160316171810",
    "rx.backpressure-lite": "registry:dt/rx.backpressure-lite#0.0.0+20160316171810",
    "rx.binding-lite": "registry:dt/rx.binding-lite#0.0.0+20160316171810",
    "rx.coincidence": "registry:dt/rx.coincidence#2.2.28+20160317120654",
    "rx.coincidence-lite": "registry:dt/rx.coincidence-lite#0.0.0+20160412135432",
    "rx.time": "registry:dt/rx.time#2.2.28+20160317120654",
    "rx.time-lite": "registry:dt/rx.time-lite#0.0.0+20160316171810",

Expected behavior:

Compiles, maybe an error...

Actual behavior:

tsc either takes a very long time and amount of resources, or runs out of memory type-checking the file.

Possibly related? #8143

I did not manage to boil down Rx.Subject and Rx.Observable to something small and self-contained, sorry about that...

[mhegazy]

i do not see this locally. looks like there is something else going on that is not clear from this snippet:

C:\test\9791>node c:\releases\1.8.10\tsc.js  --v
Version 1.8.10

C:\test\9791>node c:\releases\1.8.10\tsc.js --diagnostics --listFiles
C:/releases/1.8.10/lib.d.ts
a.ts
typings/globals/rx-lite/index.d.ts
typings/globals/rx.async-lite/index.d.ts
typings/globals/rx.backpressure-lite/index.d.ts
typings/globals/rx.binding-lite/index.d.ts
typings/globals/rx.coincidence-lite/index.d.ts
typings/globals/rx.coincidence/index.d.ts
typings/globals/rx.time-lite/index.d.ts
typings/globals/rx.time/index.d.ts
typings/globals/rx/index.d.ts
typings/index.d.ts
Files:              12
Lines:           18371
Nodes:           88860
Identifiers:     35135
Symbols:        304820
Types:           10922
Memory used:    69504K
I/O read:        0.03s
I/O write:       0.00s
Parse time:      0.23s
Bind time:       0.16s
Check time:      0.64s
Emit time:       0.03s
Total time:      1.06s

C:\test\9791>tsc --v
Version 2.1.0-dev.20160718

C:\test\9791>tsc --diagnostics --listFiles
C:/Users/mhegazy/AppData/Roaming/npm/node_modules/typescript/lib/lib.d.ts
C:/test/9791/a.ts
C:/test/9791/typings/globals/rx-lite/index.d.ts
C:/test/9791/typings/globals/rx.async-lite/index.d.ts
C:/test/9791/typings/globals/rx.backpressure-lite/index.d.ts
C:/test/9791/typings/globals/rx.binding-lite/index.d.ts
C:/test/9791/typings/globals/rx.coincidence-lite/index.d.ts
C:/test/9791/typings/globals/rx.coincidence/index.d.ts
C:/test/9791/typings/globals/rx.time-lite/index.d.ts
C:/test/9791/typings/globals/rx.time/index.d.ts
C:/test/9791/typings/globals/rx/index.d.ts
C:/test/9791/typings/index.d.ts
Files:              12
Lines:           20066
Nodes:          107159
Identifiers:     40819
Symbols:        146567
Types:           29352
Memory used:   125808K
I/O read:        0.01s
I/O write:       0.00s
Parse time:      0.27s
Bind time:       0.16s
Check time:      1.53s
Emit time:       0.03s
Total time:      1.98s

[Ptival]

Actually, it depends on me adding some type overloads that make the type-checking process go much slower.

Using just the following files:

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/324169800cb792855a96272f5d58f646e7b1d56d/rx/rx-lite.d.ts

// bug.ts
interface IWhatever { }

interface IBug<T extends IWhatever> {
  foo$: Rx.Observable<{}>;
}

export class Bug<T extends IWhatever> implements IBug<T> {
  foo$: Rx.Subject<{}>;
}

// patch.d.ts
declare namespace Rx {
  export interface Observable<T> extends IObservable<T> {
    // Overload 1
    //flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => IPromise<TResult>, thisArg?: any): Observable<TResult>;
    // Overload 2
    //flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => Array<TResult>, thisArg?: any): Observable<TResult>;
  }
}

Without uncommenting the overloads in patch.d.ts, I get the following performance:

Files:               4
Lines:           19664
Nodes:           97749
Identifiers:     37124
Symbols:         65328
Types:           19522
Memory used:    89011K
I/O read:        0.00s
I/O write:       0.00s
Parse time:      0.37s
Bind time:       0.22s
Check time:      1.67s
Emit time:       0.04s
Total time:      2.30s

When I uncomment the first overload:

Files:               4
Lines:           19664
Nodes:           97783
Identifiers:     37140
Symbols:        715336
Types:          255841
Memory used:   469343K
I/O read:        0.00s
I/O write:       0.00s
Parse time:      0.35s
Bind time:       0.17s
Check time:     15.13s
Emit time:       0.03s
Total time:     15.68s

When uncommenting the second overload:

Files:               4
Lines:           19664
Nodes:           97817
Identifiers:     37156
Symbols:       1601807
Types:          583057
Memory used:  1053833K
I/O read:        0.00s
I/O write:       0.00s
Parse time:      0.36s
Bind time:       0.16s
Check time:     33.85s
Emit time:       0.03s
Total time:     34.40s

In this setting it does not crash, but in bigger contexts, I sometimes get crashes like the following:

<--- Last few GCs --->

   48162 ms: Mark-sweep 1335.9 (1434.7) -> 1335.8 (1434.7) MB, 1896.8 / 0 ms [allocation failure] [GC in old space requested].
   50046 ms: Mark-sweep 1335.8 (1434.7) -> 1335.8 (1434.7) MB, 1884.2 / 0 ms [allocation failure] [GC in old space requested].
   51941 ms: Mark-sweep 1335.8 (1434.7) -> 1335.8 (1434.7) MB, 1895.1 / 0 ms [last resort gc].
   53823 ms: Mark-sweep 1335.8 (1434.7) -> 1335.8 (1434.7) MB, 1881.3 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3e102f8c9e31 <JS Object>
    1: objectTypeRelatedTo [/home/ptival/PeaCoq/web/node_modules/typescript/lib/tsc.js:~17954] [pc=0x1b90e25b2a95] (this=0x3e102f8e07d1 <JS Global Object>,source=0x24c523a35271 <a Type with map 0x35ca15d3ade9>,originalSource=0x24c523a35271 <a Type with map 0x35ca15d3ade9>,target=0x2297ecd0d861 <a Type with map 0x17f657daf981>,reportErrors=0x3e102f804299 <false>)
    2: isRelatedTo [/home/ptival...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0xdc783c [node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewTransitionArray(int) [node]
 6: v8::internal::TransitionArray::Allocate(v8::internal::Isolate*, int, int) [node]
 7: v8::internal::TransitionArray::Insert(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>, v8::internal::SimpleTransitionFlag) [node]
 8: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Handle<v8::internal::LayoutDescriptor>, v8::internal::TransitionFlag, v8::internal::MaybeHandle<v8::internal::Name>, char const*, v8::internal::SimpleTransitionFlag) [node]
 9: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node]
10: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [node]
11: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
14: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
15: v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode) [node]
16: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
17: 0x1b90e210961b
[1]    2320 abort      ./tsc -p . --diagnostics

[Ptival]

Here is another way in which I get the same performance degradation.

Fast:

// bar.ts
export function setup(tip$: Rx.Observable<ISentence>) {
  tip$.concatMap(s => s.completionAdded$).subscribe(x => { });
}

// foo.d.ts
interface ISentence { completionAdded$: Rx.Subject<{}>; }

declare namespace Rx {
  export interface Observable<T> extends IObservable<T> {
    flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => IPromise<TResult>, thisArg?: any): Observable<TResult>;
    flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => Array<TResult>, thisArg?: any): Observable<TResult>;
  }
}

Slow (the difference here is that I added a <S> parameter to ISentence):

// bar.ts
export function setup(tip$: Rx.Observable<ISentence<any>>) {
  tip$.concatMap(s => s.completionAdded$).subscribe(x => { });
}

// foo.d.ts
interface ISentence<S> { completionAdded$: Rx.Subject<{}>; }

declare namespace Rx {
  export interface Observable<T> extends IObservable<T> {
    flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => IPromise<TResult>, thisArg?: any): Observable<TResult>;
    flatMapLatest<TResult>(selector: (value: T, index: number, source: Observable<T>) => Array<TResult>, thisArg?: any): Observable<TResult>;
  }
}

The stack trace:

<--- Last few GCs --->

  123219 ms: Mark-sweep 1080.1 (1327.9) -> 1029.8 (1303.9) MB, 3378.6 / 0 ms [allocation failure] [GC in old space requested].
  125909 ms: Mark-sweep 1029.8 (1303.9) -> 1029.4 (1299.9) MB, 2690.2 / 0 ms [allocation failure] [GC in old space requested].
  128659 ms: Mark-sweep 1029.4 (1299.9) -> 1029.4 (1268.9) MB, 2749.7 / 0 ms [last resort gc].
  131523 ms: Mark-sweep 1029.4 (1268.9) -> 1029.4 (1259.9) MB, 2864.5 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x160e2d2c9e31 <JS Object>
    1: objectTypeRelatedTo [/home/ptival/PeaCoq/web/node_modules/typescript/lib/tsc.js:~17954] [pc=0x1b79e7cf4362] (this=0x160e2d2e07d1 <JS Global Object>,source=0x20229b355e69 <a Type with map 0x93b6743ce9>,originalSource=0x20229b355e69 <a Type with map 0x93b6743ce9>,target=0x31b0d6b32941 <a Type with map 0x93b6743ce9>,reportErrors=0x160e2d204299 <false>)
    2: isRelatedTo [/home/ptival/PeaCo...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0xdc783c [node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::HashTable<v8::internal::NameDictionary, v8::internal::NameDictionaryShape, v8::internal::Handle<v8::internal::Name> >::New(v8::internal::Isolate*, int, v8::internal::MinimumCapacity, v8::internal::PretenureFlag) [node]
 7: v8::internal::Dictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape, v8::internal::Handle<v8::internal::Name> >::EnsureCapacity(v8::internal::Handle<v8::internal::NameDictionary>, int, v8::internal::Handle<v8::internal::Name>) [node]
 8: v8::internal::Dictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape, v8::internal::Handle<v8::internal::Name> >::Add(v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails) [node]
 9: v8::internal::JSObject::AddSlowProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [node]
10: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::StoreIC::Store(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::KeyedStoreIC::Store(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node]
14: v8::internal::Runtime_KeyedStoreIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [node]
15: 0x1b79e780961b
[1]    19140 abort      ./tsc -p . --diagnostics

[mhegazy]

@vladima any insights here?

[endel]

I'm having the same error trying to compile a quite big application (+400 files, 314912 LOC)

Not sure what could be the cause.

[mhegazy]

@endel can you share a project we can debug against. we would be happy to sing any NDA needed.

[endel]

@mhegazy sure, that'd be great. Which email address should we send it?

[mhegazy]

mhegazy at microsoft dot com