Error installing python packages using `uv`

[AlejandroGomezFrieiro]

What happened?

When settings up devbox shell, setting up uv as a package to use as an installer, and then trying to use it to install numpy (or any other package) it complains with

error: Failed to install: numpy-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (numpy==2.1.3)
  Caused by: failed to create directory `/nix/store/m85llqyd79hjm4cbgx8hds2rarbslc1s-python3-3.11.2/lib/python3.11/site-packages/numpy`
  Caused by: Permission denied (os error 13)

Steps to reproduce

1. Setup a python devbox with uv
2. devbox shell
3. uv pip install numpy

Command

No response

devbox.json

{
    "$schema": "https://raw.githubusercontent.com/jetify-com/devbox/0.13.6/.schema/devbox.schema.json",
      "packages": [
        "python@3.11.2",
        "uv@latest",
        "git@latest"
      ],

    "shell": {
    "init_hook": [
      "source $VENV_DIR/bin/activate",
    ],
      "scripts": {
        "test": [
          "echo \"Error: no test specified\" && exit 1"
        ]
      }
    }
  }

Devbox version

0.13.6

Nix version

nix (Nix) 2.24.7

What system does this bug occur on?

Linux (x86-64)

Debug logs

No response

[yemaney]

I've also had the same issue, and tried to look into it. Although my error looks slightly different. Ended up resolving it by updating the UV_PYTHON environment variable to point to the python under the .venv folder created by devbox.

Explanation

There are two locations where python executable is available in a devbox project.

1. .devbox/nix/profile/default/bin/python: default UV_PYTHON
2. .venv/bin/python: symlink to python under .devbox (above)

Each of them have a related site-packages directory, where installed libraries are saved to.

The UV_PYTHON default value will mean it will try to install and save libraries to the site-packages from (1), but this directory is not writable, probably because its contents are symlinked to /nix/store/. Making sure UV_PYTHON is set to the python under .venv will allow saving the installed libraries to the site-packages from (2) which is writable.

Related sources:

• UV_PYTHON
• similar uv issue for nixos

Error

$ uv pip install numpy
Using Python 3.12.1 environment at /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1
error: The interpreter at /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1 is externally managed, and indicates the following:

  This command has been disabled as it tries to modify the immutable
  `/nix/store` filesystem.

  To use Python with Nix and nixpkgs, have a look at the online documentation:
  <https://nixos.org/manual/nixpkgs/stable/

Consider creating a virtual environment with `uv venv`.

Fix

{
  "env": {
    "UV_PYTHON": "$PWD/.venv/bin/python"
  },
  "packages": [
    "python@latest",
    "uv@latest"
  ],
  "shell": {
    "init_hook": [
      ". $VENV_DIR/bin/activate",
      "uv pip install  -r requirements.txt"
    ],
    "scripts": {
      "run_test": "python main.py"
    }
  }
}

$ devbox shell
Starting a devbox shell...
Warning: Your devbox environment may be out of date. Run refresh-global to update it.
Virtual environment directory doesn't exist. Creating new one...
Resolved 5 packages in 25ms
Installed 5 packages in 17ms
 + emoji==2.1.0
 + iniconfig==2.0.0
 + packaging==24.2
 + pluggy==1.5.0
 + pytest==8.3.4
$ echo $UV_PYTHON
/home/ubuntu/python-uv/.venv/bin/python
$ uv pip install numpy
Resolved 1 package in 108ms
Installed 1 package in 101ms
 + numpy==2.2.0

Investigating

• Regular python

$ echo $VENV_DIR
/home/ubuntu/python-uv/.venv

# follow path of python
$ which python
/home/ubuntu/python-uv/.venv/bin/python
$ ls -l /home/ubuntu/python-uv/.venv/bin/python
lrwxrwxrwx 1 ubuntu ubuntu 61 Dec 16 01:13 /home/ubuntu/python-uv/.venv/bin/python -> /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
$ ls -l /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
lrwxrwxrwx 1 ubuntu ubuntu 69 Jan  1  1970 /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python -> /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1/bin/python

# site-packages under .venv is writable
$ ls -l .venv/lib/python3.12/
total 4
drwxrwxr-x 15 ubuntu ubuntu 4096 Dec 16 02:15 site-packages

• uv

$ echo $UV_PYTHON
/home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
$ ls -l $UV_PYTHON
lrwxrwxrwx 1 ubuntu ubuntu 69 Jan  1  1970 /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python -> /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1/bin/python

# site-packages under .devbox/ is not writeable
$ ls -l  .devbox/nix/profile/default/lib/

$ ls -l  .devbox/nix/profile/default/lib/python3.12/
dr-xr-xr-x 2 ubuntu ubuntu 4096 Dec 16 02:38 site-packages

[MarkValadez]

I ran into this issue, I found that "UV_PYTHON": "$VENV_DIR/bin/python" worked more reliably. There are some cases where I still ran into issues with @yemaney's approach.

Realistically they are the same when you call the devbox shell command from where the devbox.json is located which you would if you are calling it manually. I think the discrepancy may arise when using the VSCode extension for some reason or another.