sysconfig LDPATH is not used building a library within tox #3547
Description
Issue
Building or installing a library that required a compiled binary with tox will sometimes generate an invalid binary. Done outside of tox (or virtualenv), there are not problems. Note also that there is no problem using venv.
This is to me unexpected that running something with tox result in a different result than running it outside tox in a clean environment.
pip install <library> or pip wheel <library> done within tox will build the library using the correct version of python but without using the flags used to build that python binary (as opposed to running pip install outside tox).
I understood that this is done by invoking sysconfig, but that sysconfig does not returns the proper flags within a virtual environment.
I am not sure whether the issue comes from virtualenv, or from tox using an inapropriate dependency to manage virtual environment.
In my opinion, there are no ways for tox or virtualenv to know whether the flags used to build python are relevant when building a library or not. And so forwarding them, or offering the option to forward them, should be the way to go.
In my case, my python requires a special libstdc++ to work and the path to that library is properly injected into the RPATH of the python binary. If I build a python module that also requires this library (almost all c++ libraries do), the RPATH is not properly injected and lead to the library linking with the OS version of that library.
I can work around the issue by setting:
setenv =
LDFLAGS=`python -c "import sysconfig; print(sysconfig.get_config_var('LDFLAGS'))"`
But this is less than ideal because it feels all my tox will have to do that and it works fine without tox.
Environment
Provide at least:
- OS: Unix RHEL 9, Python 3.11.8
Output of pip list of the host Python, where tox is installed
Package Version
------------------ ----------------
aiohappyeyeballs 2.6.1
aiohttp 3.12.9
aiosignal 1.3.2
apprise 1.9.3
attrs 25.3.0
auditwheel 6.2.0
black 25.1.0
build 1.2.2.post1
cachetools 5.5.2
certifi 2025.4.26
chardet 5.2.0
charset-normalizer 3.4.2
click 8.1.8
colorama 0.4.6
colorlog 6.9.0
distlib 0.3.9
filelock 3.17.0
frozenlist 1.6.2
gcovr 8.3
hvac 2.3.0
idna 3.10
isort 6.0.1
Jinja2 3.1.6
libclang 18.1.1
lxml 5.3.1
Markdown 3.8
MarkupSafe 3.0.2
multidict 6.4.4
mypy_extensions 1.1.0
oauthlib 3.2.2
packaging 24.2
patchelf 0.15.0.0
pathspec 0.12.1
pip 25.1.1
platformdirs 4.3.6
pluggy 1.5.0
propcache 0.3.1
pyelftools 0.32
Pygments 2.19.1
pymsteams 0.2.5
pyproject-api 1.9.0
pyproject_hooks 1.2.0
PyYAML 6.0.2
quickfix 1.15.1
requests 2.32.3
requests-oauthlib 2.0.0
setuptools 80.9.0
tox 4.26.0
urllib3 2.4.0
virtualenv 20.31.2
wheel 0.45.1
yarl 1.20.0Output of running tox
Output of tox -rvv
py: 166 I find interpreter for spec PythonSpec(path=/opt/tools/python-3.11.8/bin/python3.11) [virtualenv/discovery/builtin.py:76]
py: 167 D filesystem is case-sensitive [virtualenv/info.py:27]
py: 167 I proposed PythonInfo(spec=CPython3.11.8.final.0-64, exe=/opt/tools/python-3.11.8/bin/python3.11, platform=linux, version='3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py: 167 D accepted PythonInfo(spec=CPython3.11.8.final.0-64, exe=/opt/tools/python-3.11.8/bin/python3.11, platform=linux, version='3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:85]
py: 189 I create virtual environment via CPython3Posix(dest=/home/user/tmp/tmp/.tox/py, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:52]
py: 190 D create folder /home/user/tmp/tmp/.tox/py/bin [virtualenv/util/path/_sync.py:14]
py: 190 D create folder /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages [virtualenv/util/path/_sync.py:14]
py: 192 D write /home/user/tmp/tmp/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
py: 192 D home = /opt/tools/python-3.11.8/bin [virtualenv/create/pyenv_cfg.py:40]
py: 192 D implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
py: 192 D version_info = 3.11.8.final.0 [virtualenv/create/pyenv_cfg.py:40]
py: 192 D virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
py: 192 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
py: 192 D base-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 192 D base-exec-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 192 D base-executable = /opt/tools/python-3.11.8/bin/python3.11 [virtualenv/create/pyenv_cfg.py:40]
py: 193 D symlink /opt/tools/python-3.11.8/bin/python3.11 to /home/user/tmp/tmp/.tox/py/bin/python [virtualenv/util/path/_sync.py:34]
py: 193 D create virtualenv import hook file /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:93]
py: 194 D create /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:96]
py: 194 D ============================== target debug ============================== [virtualenv/run/session.py:54]
py: 194 D debug via /home/user/tmp/tmp/.tox/py/bin/python /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:215]
py: 194 D {
"sys": {
"executable": "/home/user/tmp/tmp/.tox/py/bin/python",
"_base_executable": "/opt/tools/python-3.11.8/bin/python3.11",
"prefix": "/home/user/tmp/tmp/.tox/py",
"base_prefix": "/opt/tools/python-3.11.8",
"real_prefix": null,
"exec_prefix": "/home/user/tmp/tmp/.tox/py",
"base_exec_prefix": "/opt/tools/python-3.11.8",
"path": [
"/opt/tools/python-3.11.8/lib/python311.zip",
"/opt/tools/python-3.11.8/lib/python3.11",
"/opt/tools/python-3.11.8/lib/python3.11/lib-dynload",
"/home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]",
"makefile_filename": "/opt/tools/python-3.11.8/lib/python3.11/config-3.11-x86_64-linux-gnu/Makefile",
"os": "<module 'os' (frozen)>",
"site": "<module 'site' (frozen)>",
"datetime": "<module 'datetime' from '/opt/tools/python-3.11.8/lib/python3.11/datetime.py'>",
"math": "<module 'math' from '/opt/tools/python-3.11.8/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so'>",
"json": "<module 'json' from '/opt/tools/python-3.11.8/lib/python3.11/json/__init__.py'>"
} [virtualenv/run/session.py:55]
py: 222 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/user/.local/share/virtualenv) [virtualenv/run/session.py:59]
py: 225 D install setuptools from wheel /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/setuptools-80.3.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:51]
py: 226 D install pip from wheel /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/pip-25.1.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:51]
py: 227 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/pip [virtualenv/util/path/_sync.py:42]
py: 228 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/distutils-precedence.pth to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:42]
py: 229 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/_distutils_hack to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:42]
py: 230 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/pkg_resources to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/pkg_resources [virtualenv/util/path/_sync.py:42]
py: 238 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/setuptools [virtualenv/util/path/_sync.py:42]
py: 342 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.dist-info to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/pip-25.1.1.dist-info [virtualenv/util/path/_sync.py:42]
py: 344 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.virtualenv to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/pip-25.1.1.virtualenv [virtualenv/util/path/_sync.py:42]
py: 346 D generated console scripts pip-3.11 pip3 pip3.11 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:45]
py: 355 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools-80.3.1.dist-info to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/setuptools-80.3.1.dist-info [virtualenv/util/path/_sync.py:42]
py: 357 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools-80.3.1.virtualenv to /home/user/tmp/tmp/.tox/py/lib/python3.11/site-packages/setuptools-80.3.1.virtualenv [virtualenv/util/path/_sync.py:42]
py: 357 D generated console scripts [virtualenv/seed/embed/via_app_data/pip_install/base.py:45]
py: 357 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:65]
py: 360 D write /home/user/tmp/tmp/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
py: 361 D home = /opt/tools/python-3.11.8/bin [virtualenv/create/pyenv_cfg.py:40]
py: 361 D implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
py: 361 D version_info = 3.11.8.final.0 [virtualenv/create/pyenv_cfg.py:40]
py: 361 D virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
py: 361 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
py: 361 D base-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 361 D base-exec-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 361 D base-executable = /opt/tools/python-3.11.8/bin/python3.11 [virtualenv/create/pyenv_cfg.py:40]
py: OK (0.20 seconds)
congratulations :) (0.26 seconds)
[02:13:13]user@myhost:~/tmp/tmp$ cd
[02:13:38]user@myhost:~$ tox -rvv
ROOT: No tox.ini or setup.cfg or pyproject.toml or tox.toml found, assuming empty tox.ini at /home/user
py: 166 I find interpreter for spec PythonSpec(path=/opt/tools/python-3.11.8/bin/python3.11) [virtualenv/discovery/builtin.py:76]
py: 167 D filesystem is case-sensitive [virtualenv/info.py:27]
py: 167 I proposed PythonInfo(spec=CPython3.11.8.final.0-64, exe=/opt/tools/python-3.11.8/bin/python3.11, platform=linux, version='3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py: 167 D accepted PythonInfo(spec=CPython3.11.8.final.0-64, exe=/opt/tools/python-3.11.8/bin/python3.11, platform=linux, version='3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:85]
py: 189 I create virtual environment via CPython3Posix(dest=/home/user/.tox/py, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:52]
py: 189 D create folder /home/user/.tox/py/bin [virtualenv/util/path/_sync.py:14]
py: 190 D create folder /home/user/.tox/py/lib/python3.11/site-packages [virtualenv/util/path/_sync.py:14]
py: 191 D write /home/user/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
py: 191 D home = /opt/tools/python-3.11.8/bin [virtualenv/create/pyenv_cfg.py:40]
py: 191 D implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
py: 191 D version_info = 3.11.8.final.0 [virtualenv/create/pyenv_cfg.py:40]
py: 191 D virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
py: 191 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
py: 191 D base-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 191 D base-exec-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 191 D base-executable = /opt/tools/python-3.11.8/bin/python3.11 [virtualenv/create/pyenv_cfg.py:40]
py: 191 D symlink /opt/tools/python-3.11.8/bin/python3.11 to /home/user/.tox/py/bin/python [virtualenv/util/path/_sync.py:34]
py: 192 D create virtualenv import hook file /home/user/.tox/py/lib/python3.11/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:93]
py: 192 D create /home/user/.tox/py/lib/python3.11/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:96]
py: 192 D ============================== target debug ============================== [virtualenv/run/session.py:54]
py: 193 D debug via /home/user/.tox/py/bin/python /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:215]
py: 193 D {
"sys": {
"executable": "/home/user/.tox/py/bin/python",
"_base_executable": "/opt/tools/python-3.11.8/bin/python3.11",
"prefix": "/home/user/.tox/py",
"base_prefix": "/opt/tools/python-3.11.8",
"real_prefix": null,
"exec_prefix": "/home/user/.tox/py",
"base_exec_prefix": "/opt/tools/python-3.11.8",
"path": [
"/opt/tools/python-3.11.8/lib/python311.zip",
"/opt/tools/python-3.11.8/lib/python3.11",
"/opt/tools/python-3.11.8/lib/python3.11/lib-dynload",
"/home/user/.tox/py/lib/python3.11/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.11.8 (main, Mar 7 2025, 12:30:05) [GCC 12.3.0]",
"makefile_filename": "/opt/tools/python-3.11.8/lib/python3.11/config-3.11-x86_64-linux-gnu/Makefile",
"os": "<module 'os' (frozen)>",
"site": "<module 'site' (frozen)>",
"datetime": "<module 'datetime' from '/opt/tools/python-3.11.8/lib/python3.11/datetime.py'>",
"math": "<module 'math' from '/opt/tools/python-3.11.8/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so'>",
"json": "<module 'json' from '/opt/tools/python-3.11.8/lib/python3.11/json/__init__.py'>"
} [virtualenv/run/session.py:55]
py: 221 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/user/.local/share/virtualenv) [virtualenv/run/session.py:59]
py: 223 D install pip from wheel /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/pip-25.1.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:51]
py: 223 D install setuptools from wheel /opt/tools/python-3.11.8/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/setuptools-80.3.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:51]
py: 226 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip to /home/user/.tox/py/lib/python3.11/site-packages/pip [virtualenv/util/path/_sync.py:42]
py: 227 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/distutils-precedence.pth to /home/user/.tox/py/lib/python3.11/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:42]
py: 227 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/_distutils_hack to /home/user/.tox/py/lib/python3.11/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:42]
py: 228 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/pkg_resources to /home/user/.tox/py/lib/python3.11/site-packages/pkg_resources [virtualenv/util/path/_sync.py:42]
py: 235 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools to /home/user/.tox/py/lib/python3.11/site-packages/setuptools [virtualenv/util/path/_sync.py:42]
py: 313 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.dist-info to /home/user/.tox/py/lib/python3.11/site-packages/pip-25.1.1.dist-info [virtualenv/util/path/_sync.py:42]
py: 315 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.virtualenv to /home/user/.tox/py/lib/python3.11/site-packages/pip-25.1.1.virtualenv [virtualenv/util/path/_sync.py:42]
py: 317 D generated console scripts pip3.11 pip-3.11 pip3 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:45]
py: 334 D copy directory /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools-80.3.1.dist-info to /home/user/.tox/py/lib/python3.11/site-packages/setuptools-80.3.1.dist-info [virtualenv/util/path/_sync.py:42]
py: 336 D copy /home/user/.local/share/virtualenv/wheel/3.11/image/1/CopyPipInstall/setuptools-80.3.1-py3-none-any/setuptools-80.3.1.virtualenv to /home/user/.tox/py/lib/python3.11/site-packages/setuptools-80.3.1.virtualenv [virtualenv/util/path/_sync.py:42]
py: 337 D generated console scripts [virtualenv/seed/embed/via_app_data/pip_install/base.py:45]
py: 337 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:65]
py: 340 D write /home/user/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
py: 340 D home = /opt/tools/python-3.11.8/bin [virtualenv/create/pyenv_cfg.py:40]
py: 340 D implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
py: 340 D version_info = 3.11.8.final.0 [virtualenv/create/pyenv_cfg.py:40]
py: 340 D virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
py: 340 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
py: 340 D base-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 340 D base-exec-prefix = /opt/tools/python-3.11.8 [virtualenv/create/pyenv_cfg.py:40]
py: 340 D base-executable = /opt/tools/python-3.11.8/bin/python3.11 [virtualenv/create/pyenv_cfg.py:40]
py: OK (0.18 seconds)
congratulations :) (0.24 seconds)Minimal example
[tox]
skipsdist = true
[testenv:doit]
skip_install = true
setenv =
LDFLAGS=`python3 -c "import sysconfig; print(sysconfig.get_config_var('LDFLAGS'))"`
commands =
pip3 wheel <library>