Nix integration should use GC roots

[domenkozar]

Usual scenario:

1. build stack project with Nix
2. sometime later you run nix-store --gc or nix-collect-garbage
3. you run into following error:

/home/user/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2: streamingProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

This means that nix libraries that the file was compiled with are missing from nix store.

Best way to prevent this is to create Nix GC roots. Also stack could double check that those libraries are still intact.

[domenkozar]

Another quite common scenario: people cache ~/.stack on CI, but not /nix/store. Not sure we can do much here, stack could have a better message though.

[mgsloan]

Contributions appreciated! It'd be great to have someone that is familiar with / uses nix work on this. Is this an issue with foreign libraries or plain haskell libraries getting GCed? If the latter, could a blunt approach work? Make each package DB into a GC root.

[domenkozar]

Another solution (which might be in general more reliable) is that stack would rebuild Cabal-simple if it can't execute it.

[mmzx]

I've just closed #3999, perhaps was caused by the same thing nicely outlined here. Do you think this issue could persist? If so, what would be the way to reproduce it currently?

[roberth]

@mmzx This is not just a packaging issue. The problem is that Stack creates libraries and executables that reference nix store paths without creating the necessary garbage collection roots to prevent those paths from being deleted in the future.
This also means that the problem is bigger than just the setup exe cache. It applies to every reference to the nix store in every library or executable produced by stack.

Steps to reproduce the problem are in the issue description.

[roberth]

Another solution (which might be in general more reliable) is that stack would rebuild Cabal-simple if it can't execute it.

This is only sufficient for binaries that are built by stack and called by stack. The problem also affects libraries and executables in ~/.stack

[bdesham]

The changelog for Stack 1.3.0 includes this item:

Stack/nix: Dependencies can be added as nix GC roots, so they are not removed when running nix-collect-garbage

Likewise, the Stack manual details a related configuration option:

# false by default. Whether to add your nix dependencies as nix garbage
# collection roots. This way, calling nix-collect-garbage will not remove
# those packages from the nix store, saving you some time when running
# stack build again with nix support activated.
# This creates a `nix-gc-symlinks` directory in the project `.stack-work`.
# To revert that, just delete this `nix-gc-symlinks` directory.
add-gc-roots: false

Has this actually been implemented? If so, I believe this issue can be closed, right?

(I’m having trouble with this configuration option—I’m setting it to true but I’m not getting a nix-gc-symlinks directory—but that would be a separate ticket.)

[domenkozar]

There's #4673, but maybe this issue should be about turning them on by default.