Propagating build inputs

[jaen]

My use case is being able to run bundle install in the shell for projects using Ruby gems with native dependencies that need to be compiled with gcc (say, libxml or Postgres client). This works when I specify those pacakges as buildInputs in pkgs.mkShell, but there's no option to do so with devshell and even when I hacked the source a bit to pass it down to buildEnv it doesn't help either – the culprit seems to be the list of environment variables passed down to the env (it's lacking the compiler-related envvars).

I'm not sure what's the best direction to take here (I'm a bit of a newb when it's coming to using nix as something other than a glorified ansible for my home infrastructure) but I think it's an useful use-case to cover.

[yusdacra]

I'm also having this issue. For example, putting glib in buildInputs in mkShell makes glib's pkg-config files available, but with mkDevShells packages this doesn't work. Or putting mpv in packages then trying to link to libmpv also won't work.

[zimbatm]

I had to remove the pkgs.stdenv machinery in order to re-build a clean user experience. This means that all of the standard nix-shell hooks are missing.

For now, the best option is to either use pkgs.mkShell, or to extend the devshell with the proper environment variables. I'm working on adding those back as proper options.

[blaggacao]

I also wondered if it is generally a good idea to put (proper) project build dependencies into the devshell. An alternative place that would come to my mind would be a proper build / release nix infrastructure (outside of devshell) and then a little wrapper which would provide a commamd to the devshell (why not wrap nix build for that purpose?) And also a package that exposes the environment.

Not sure at all of this separation of concern is warranted, for sure it might be impractical of a sort. On the other hand one would probably also want those things be working also on stock CI (aka nix build / nix check).

I'll remain very interested to hear back from your thoughts on this.

[yusdacra]

I also wondered if it is generally a good idea to put (proper) project build dependencies into the devshell. An alternative place that would come to my mind would be a proper build / release nix infrastructure (outside of devshell) and then a little wrapper which would provide a commamd to the devshell (why not wrap nix build for that purpose?) And also a package that exposes the environment.

I usually seperate build dependencies, build environment variables etc. stuff into a common.nix file, and import that file in my build and shell derivations. For now I think this works good enough. However I would like to be able to just use my build derivation's environment to also develop, which would remove the need for a common.nix.

[zimbatm]

I have two sides to this:

Ideally, the devshell environment shouldn't be too fat. The issue is that each additional package makes it more expensive to load a fresh environment as Nix has to load a bunch of dependencies from the cache. Monorepos can quickly blow to multiple GB of data to download. This is a usability issue. Once everything is loaded into the /nix/store things go faster, but updating nixpkgs is not as rare as one might think. This also happens when switching development branches or bisecting old builds.

In order to solve this, I am looking to build an incremental Nix builder (yet another project). Other tools like Bazel might also be used. The advantage is that the initial pull becomes smaller, and then only fetch the subset of the tree you are interested in.

But we have to be pragmatic. In practice, LSP servers tend to need to have access to their respective compiler/interpreter. And the third-party libraries. Editors also want to be able to find the compiler to do their own checks. The story is not great there.

[zimbatm]

I forgot to say that there is also #6 which was an experiment to propagate all the build inputs of a derivation to packages. The solution is not complete as it doesn't set LD_INCLUDE_PATH and friends.

[yusdacra]

I forgot to say that there is also #6 which was an experiment to propagate all the build inputs of a derivation to packages. The solution is not complete as it doesn't set LD_INCLUDE_PATH and friends.

This sounds pretty interesting, since you could "just" have a dev env for that package by propagating the build inputs. It would also solve "using build derivations env". This also makes me think of auto-generated dev shells, which would be ideal, although it may not be possible for every build process.

[zimbatm]

That was one of the motivations for the CLI. At some point, I want to add some heuristics to devshell init so it can detect the project structure and infer some good defaults based on that.

Most current users are probably familiar with Nix already and know how to build derivations. The other types of users I want to attract are only going to use devshell to bring in the development dependencies, and then use the language native tools to build their project.

[bbigras]

Any progress or workaround for this? I would like to use devshell with Rust projects needing some libs with buildInputs.

[zimbatm]

@bbigras I added the language.c module to handle C dependencies.
With a bit of nix code is should be possible to inject those on both sides.

[bbigras]

With a bit of nix code is should be possible to inject those on both sides.

Do you mean adding the same code into rust.nix?

[Builditluc]

Can someone provide an example of how to use buildInputs with rust?