Return memory to OS after old space GC collection

[mkustermann]

We use the DartVM for running the fletchc compiler. It uses isolates for compilation. Running this on a memory constrained ARM chromebook with 4 cores sometimes causes us to run out of memory (1 main isolate and 4 worker isolates).

After doing a GC via observatory using the allocation profile UI, it seems that the DartVM still uses 1050MB from the OS even though the used memory (accumulated over all isolates) is just 245MB. That means it seems to be reserving 4 times as much memory from the OS as it actually uses.

cc: @iposva-google @peter-ahe-google @madsager

[iposva-google]

Please send reproduction instructions. Thanks!

[mkustermann]

It should be possible to reproduce this with the following commands:

$ mkdir checkout
$ cd checkout
$ gclient config https://github.com/dart-lang/fletch.git
$ gclient sync
$ cd fletch
fletch $ ./tools/test.py -cfletchc -rfletchvm -j4
<wait until done - takes a while>
fletch $ pgrep -f driver_main.dart
27497
fletch $ export DPID=27497
fletch $ cat /proc/$DPID/fd/1 | grep Observat &!
fletch $ kill -SIGQUIT $DPID
Observatory  listening on http://127.0.0.1:47917

Clicking on "GC" button for all isolates in the observatory allocation profile and one should see that the VM uses significantly more memory from the OS than it actually needs to.

It is also possible to run the corresponding Dart VM manually. One can obtain the command via pgrep -a -f driver_main.dart. Running this command manually in one terminal and running ./tools/test.py in another terminal with --kill-persistent-process=0 as an additional argument.

[mkustermann]

For some more context, if we run OOM, we hit a FATAL() in the VM - where the TODO is placed:

runtime/vm/scavenger.cc"

454 SemiSpace* Scavenger::Prologue(Isolate* isolate, bool invoke_api_callbacks) {
  ...
463   if (to_ == NULL) {
464     // TODO(koda): We could try to recover (collect old space, wait for another
465     // isolate to finish scavenge, etc.).
466     FATAL("Out of memory.\n");
467   }
 ...
473 }

[azenla]

This is something I have noticed too. I do a lot of logic on startup, and a lot less after startup. My old space heap grows large, but then never shrinks, however I notice that it never shrinks in both Observatory and in the OS.

[rmacnak-google]

The VM returns memory to the OS when every object on a page if no longer reachable. If the fraction of old space that is used is low, it is likely fragmentation.

#30978