"pub serve" running out of memory

[DartBot]

<img src="https://avatars.githubusercontent.com/u/648527?v=3" align="left" width="96" height="96"hspace="10"> Issue by Fox32
Originally opened as dart-lang/sdk#13848

---

What steps will reproduce the problem?

1. Run "pub serve"
2. Work some time while "pub serve" runs in background (~20min?)

What is the expected output? What do you see instead?
dart.exe crashes with the following output:

vm\thread_pool.css:233: error: Could not start worker thread: result = 12.

What version of the product are you using? On what operating system?
Windows 8 64 bit
Dart Editor version 0.7.6_r28108
Dart SDK version 0.7.6.4_r28108

Please provide any additional information below.
I'm not really doing anything, sometimes it crash while I'm not even on my computer.

The project contains many files, does it spawn a worker per file? I also noticed that it has high CPU usage, but that could be the file watcher using polling atm, if I see it right.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/5449880?v=3" align="left" width="48" height="48"hspace="10"> Comment by iposva-google

---

How many isolates are you creating as part of "pub serve"?

---

Set owner to @munificent.
Added Area-Pub, NeedsInfo labels.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

It will create one isolate per transformer plug-in on startup. Aside from that, I could be wrong, but I don't think it creates other isolates over time while running.

Adding Nathan.

---

cc @nex3.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Added this to the M8 milestone.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/648527?v=3" align="left" width="48" height="48"hspace="10"> Comment by Fox32

---

I don't use any additional transformers, only the build in ones.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Bouncing this over to IO.

Søren, could this be an internally created worker threads?

---

Set owner to @sgjesse.
Removed Area-Pub label.
Added Area-IO label.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

"pub serve" creates two isolates per transformer (the second exists to work around issue dart-lang/sdk#12617), and no isolates for anything other than transformers. The number of isolates should not increase after the server has started.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/22043?v=3" align="left" width="48" height="48"hspace="10"> Comment by skabet

---

Can you provide some more info on what package you run 'pub serve' for? Is it any package, or does it have to be of some size? Is it a package you can share? Does anything helps trigger it faster?

I landed some changes to threads on Windows last week. Would it be possible for you to try out a bleeding-edge binary?

Thanks!

• Anders

[DartBot]

<img src="https://avatars.githubusercontent.com/u/648527?v=3" align="left" width="48" height="48"hspace="10"> Comment by Fox32

---

I only tested my own package, by can test with a new package later (I can also try the bleeding edge build) but it cost some time.
My package is a web package (not using polymer) with 15k lines in 156 files. I have around 2k 64x64 png files in my asset folder (~9mb). I'm not using all of them at the moment, but later a sprite sheet transformer would be nice. This seems to cause a long start up time of pub serve. I see how all png files are listed if I run it in verbose mode.
 
Sometime it takes 20 minutes, sometimes 2 hours. Sometimes it happens while I'm developing and changing files of the project in the Dart Editor. But it also happens while I'm not an the computer.

I could share the project, but only via email to keep it private.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/648527?v=3" align="left" width="48" height="48"hspace="10"> Comment by Fox32

---

The problem is still occurrences in the latest build, not sure if that contains the changes you mentioned:

Dart Editor version 0.8.1_r28355
Dart SDK version 0.8.1.2_r28355

[DartBot]

<img src="https://avatars.githubusercontent.com/u/22043?v=3" align="left" width="48" height="48"hspace="10"> Comment by skabet

---

I've done a bit of debugging, and I found the following information:

1. It's using ~20-30 threads
2. It's using 400+ open handles.
3. It's maxing out one of my CPUs.
4. Slowly climbing memory, at 700 MB after ~10 min.

This is when it's just idling in the background, without a browser connected to it.

All of these items are issues that should be fixed in pub serve.

Now, the issues you see (vm\thread_pool.cc:233: error: Could not start worker thread: result = 12.) actually means that we have run out of memory. The VM currently don't track external memory, so it's likely that it's the lack of GCs that makes it go out of memory from all the external data 'pub serve' loads. This have to be fixed in the VM (I'll open a new bug for that).

Reassigning to pub.

---

Set owner to @munificent.
Removed Area-IO label.
Added Area-Pub, Accepted labels.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Now, the issues you see (vm\thread_pool.cc:233: error: Could not start worker thread: result = 12.) actually means that we have run out of memory.

Ah, that's a helpful bit of data. There's very possibly stuff that pub, barback, or the watcher package that we could change to alleviate this. We haven't done any optimization work here yet.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

Removed Priority-Unassigned label.
Added Priority-High label.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

Changing the summary to make this an umbrella issue for various pub-related memory issues.

---

Changed the title to: ""pub serve"/"pub build" running out of memory".

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

Marked this as being blocked by dart-archive/barback#15, #680.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

I believe the "pub build" side of this is fixed with issue #680.

---

Changed the title to: ""pub serve" running out of memory".

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

Marked this as being blocked by dart-lang/tools#1685.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

Marked this as being blocked by #682.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

I'm bouncing this to you because we're hoping the watcher patch will help here. (Though that's only going to help Linux for now.)

---

Set owner to @nex3.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Siggy brought up another possible culprit here: dart2js.

I know it consumes a lot of memory, but I'm hoping it frees that all up when it's done. Dart2js is mostly synchronous, so there probably isn't too much overlap and simultaneous dart2js work going on in parallel, but there is some overlap during source file loading.

It's possible that we've got a couple of dart2js compilations going concurrently and they are eating up a ton of memory.