Slow compilation issues (40-50s) - --skipLibCheck not working #41517
Comments
|
@amcasey tagging you both @RyanCavanaugh in this as I saw you from other threads with similar/slow compilation issues |
|
You could try updating your TypeScript version: https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-9.html#speed-improvements |
|
Upgrading to 3.9.7 results in basically the same speed: |
RyanCavanaugh
added
the
Needs Investigation
|
As an FYI, if you need more information please let me know and I'll be happy to provide. The only problem is that I cannot post the project, as it's a private/institutional repo. So if there is any way of providing obfuscated data about the project that will help troubleshooting please let me know |
|
@NoPhaseNoKill I wrote up some instructions for how to gather a trace that will identify which parts of your program are taking the longest to compile: https://github.com/microsoft/TypeScript/wiki/Performance-Tracing Ideally, you'd share the resulting logs, but one of our goals in adding this functionality was to be able to work with users who can't share source. I'm hoping it will be enough for you to identify the problem and either fix it on your end or provide us a reduced repro (e.g. calling the same npm module that's giving you problems). It's still a pretty new tool (and doc), so ping me if you get stuck and I'll do what I can to unblock you (except tomorrow, when I'll be away). Thanks! |
|
Anecdotally, from your huge subtype cache, I'm guessing you have a huge array of object literals somewhere (the trace should give you a file and position). If that's the case, my usual workaround is to add I suspect the node_modules .d.ts files are a red herring. The short explanation is that the parts consumed by the code you're compiling will be checked (because the compiler needs to know about it to check your code) and the others won't. On their own, .d.ts files rarely contribute to check time - the cost is nearly always at the use-site. |
|
One last point: I didn't see anything in your comment about your findings from |
|
Yeah I didn't end up reading the CPU profile. Just currently running the performance tracing atm - so will get back to you ASAP. What would an example of this be? |
|
When trying to open the trace.json file - nothing is displaying (even though it's got the fact it's opened that file). Any ideas? edit: The trace file is like 1.5GB - got a feeling it's maybe crashing/too big? |
|
I'm more than happy to send the trace.json - as it doesn't include anything specific about the source code. How would you like me to upload this? edit: I believe I can share these via OneDrive. Standby for links upon uploading |
|
There's actually a few pieces of data in here which the company considers 'sensitive'. So I'm going to go through and obfuscate it. This should be done next Monday - so I'll post here with the links when it's good to go :) |
|
Do let me know what you meant about the object literals though. We do have a large file which contains maybe 700/800 lines of object literals - so would be interested to see how we tackle this workaround considering we're assigning them to multiple consts etc. Sorry for the constant posting, just brain dumping as I find out more info. Really appreciate the help :) |
|
Ok I was able to trace the bug to the smallest possible repo. Please let me know if you need any further information @amcasey Please see: https://github.com/NoPhaseNoKill/ts-slow-compilation-example/tree/main/src When line 21 in the IconButton is commented out we get perf of: However, when we're compiling the code we get nearly a 20x in compile time: |
|
@NoPhaseNoKill Awesome! Thanks for the reduced repro! I'll dig in. Was |
|
On pattern I've seen in a number of slow-to-compile projects is something like this: const widgets: Widget[] = [
{
kind: "widget001"
// ...
},
// ...
{
kind: "widget999"
// ...
},
];The mitigation I usually use is to add a type assertion to the first list element: {
kind: "widget001"
// ...
} as Widget,(In rare cases, that causes problems too, in which case I use |
|
So I managed to fix the issue by doing this: The generate trace for this instance was not helpful, as the log file was coming out at at a whopping 1.5GB lol. Chrome basically was like "nope I ain't having that". I profiled it by myself, by deleting directories/files to narrow it down - which lead me to that line of code. The one specific difference with this bug (maybe as opposed to others of similar nature) - is that the massive object literal is coming from a third party library (semantic-ui-react in this case). Not sure if this makes a difference when fixing the issue, but something to keep in mind when you guys roll out a fix. edit: formatting |
|
I have to admit I've never seen a trace like that before. Hopefully, I'll have some concrete details for you tomorrow. |
|
@NoPhaseNoKill Is that code correct? Clearly the compiler shouldn't take so long (I think it's missing a complexity check), but I don't really understand how the type of Edit: Ah, JavaScript - of course you can attach whatever properties you want to a string... |
|
Here's a simpler repro: import { SemanticICONS } from "semantic-ui-react";
type Stuff = { x: number; } | { y: string; } | { z: boolean; };
declare const icon: SemanticICONS & Stuff;
const name: SemanticICONS = icon;For convenience, let's say So, what next?
|
|
(My react experience is limited, so this may be a very dumb suggestion.) Maybe you wanted something like this? import React from "react";
import {
Button,
ButtonProps,
Icon,
IconProps,
} from "semantic-ui-react";
interface Props {
icon: IconProps;
};
//doesn't even need to be used
class IconButton extends React.Component<ButtonProps & Props> {
render() {
return (
<Button>
{/*comment out line 21 to see difference in performance*/}
<Icon name={this.props.icon.name} />
</Button>
);
}
}
export default IconButton;It seems like |
|
Yeah the code I posted was just a demonstration of what caused the bug. The typing (as you've mentioned) was absolutely a little odd - and needed to be fixed. I didn't post this due to it being a private repo.
edit: Really appreciate the help with debugging this/the time you've put in btw. |
If their multiplied size is greater than 1E7 (chosen based on the repro in microsoft#41517), then we'll expend a large amount of time and memory comparing them, so alert the user instead. Fixes microsoft#41517
|
@amcasey @NoPhaseNoKill I'm experiencing the same issue and am also using semantic-ui-react. My trace file is also 1.5GB. Do either of you have any tricks to reduce the size of the file? @amcasey mentioned post-processing to remove fast entries - any tips or tools for doing that? |
|
Performance tool in its current state didn't help me at all unfortunately. The best way to debug this (albeit tedious) - was to remove 50% of your repo at a time (ie delete) - recompile, and see whether there was a noticeable difference. Once you figure out which side of the repo is causing the issue, do a 50% delete on that and so forth. This resulted in it narrowing down to a singular file, and you should then be able to apply the above fix (from previous posts) to solve your issue |
|
@lafritay I have a local script that I've cleaned up and just have to post somewhere. Before we try that though, if you're using the same library, it seems likely that you're hitting a similar issue, which the compiler should now detect. Edit: I just noticed that the PR isn't merged, so installing nightly won't help. Instead, you can grab a temp build from here. I'll work on getting it merged for real. |
|
@amcasey I was able to find the issue using the process @NoPhaseNoKill described so I think we're good here. Thanks for the work on the diagnostics though - it's really helpful. |
If their multiplied size is greater than 1E7 (chosen based on the repro in microsoft#41517), then we'll expend a large amount of time and memory comparing them, so alert the user instead. Fixes microsoft#41517
|
@amcasey Something that was slow. Removing this intersection got us 20 seconds of build time back:
|
|
@lafritay Thanks for the update! If you use this build of TS, I think you should be able to keep that code without losing build time. Please let me know if you have a chance to confirm. |
* Detect comparisons between large unions or intersections If their multiplied size is greater than 1E6 (chosen based on the repro in #41517), then we'll expend a large amount of time and memory comparing them, so record a trace event. Related to #41517 * Make an exception for primitive union comparisons * Address PR feedback * Pick up baseline change from master * Eliminate diagnostic and only trace * Don't check reportErrors
TypeScript Version: 3.8.3
Search Terms: --skipLibCheck slow compilation tsc
Code
Running tsc --listFiles --generateCpuProfile profile --skipLibCheck from the command line results in having the following output - which appears to be slowing compilation significantly. It appears as though it's including multiple .d.ts files from my @types node_modules in the compilation. What sort of time should I be expecting this amount of files to compile in? Any help reducing the compilation time would also be greatly appreciated.
my tsconfig is as follows:
with my project folder structure being:
Expected behavior:
Should not type-check any of the files inside node_modules during compilation
Actual behavior:
Appears to be including node_modules/@types during compilation - which is causing a compilation time of ~50s for project.
Related Issues:
#37635
#36103
The text was updated successfully, but these errors were encountered: