[BUG] tox4 invoked in a runpy mode (python -m) thinks there's no config under windows when the interpreter also runs in isolation (-I) and the config contains a unicode character

[webknjaz]

Issue

It seems like --skip-pkg-install influences the config detection in tox somehow, making it think there's no config present. I'm trying to handle the compatibility quirks in the CI of https://github.com/ansible-community/ansible-pygments — a fairly simple project with a simple testing setup. One thing that may be a bit less common is building the dists and then using them via --installpkg.
Once I've added a workaround for #2442, jobs running Linux or macOS succeeded running tests. But Windows was failing.
I looked inside and discovered some weird logs — it does not run any envs and succeeds with that.

First off, I was lucky to notice this because the tox invocation returns a successful status while not running any env at all — it only prints out a warning saying No tox.ini or setup.cfg or pyproject.toml found, assuming empty tox.ini and exits. Good that I had a subsequent step that actually failed because of a missing junit file. In the CI setups that have the tox invocation as the last step or the following steps that don't depend on it succeeding, the maintainers won't notice that nothing actually runs, making the CI expectations misleading. Well, if they don't also have some sort of code coverage that would notice win-specific gaps, that is.
I also have the CI re-run failing invocations with maximum verbosity for tox and pytest. So maybe that output will shed some light on what's happening — I haven't checked the source code to debug this because I don't have a Windows machine in my possession.

Environment

Provide at least:

• OS: windows-latest @ GHA
• pip list of the host Python where tox is installed: cachetools-5.2.0 chardet-5.1.0 colorama-0.4.6 distlib-0.3.6 filelock-3.8.2 packaging-22.0 platformdirs-2.6.0 pluggy-1.0.0 pyproject-api-1.2.1 tox-4.0.8 virtualenv-20.17.1 (this is from the last line in pip install, my CI setup doesn't have a pip list in it)

Output of running tox

$ python -m tox --parallel auto --parallel-live --skip-missing-interpreters false -vvvvv --skip-pkg-install -- --no-cov -vvvvv --lf
ROOT: 218 D setup logging to NOTSET on pid 2400 [tox\report.py:221]
ROOT: 296 W No tox.ini or setup.cfg or pyproject.toml found, assuming empty tox.ini at D:\a\ansible-pygments\ansible-pygments [tox\config\source\discover.py:77]
python: 328 I find interpreter for spec PythonSpec() [virtualenv\discovery\builtin.py:56]
python: 343 I proposed PythonInfo(spec=CPython3.11.0.final.0-64, exe=C:\hostedtoolcache\windows\Python\3.11.0\x64\python.exe, platform=win32, version='3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]', encoding_fs_io=utf-8-cp1252) [virtualenv\discovery\builtin.py:63]
python: 343 D accepted PythonInfo(spec=CPython3.11.0.final.0-64, exe=C:\hostedtoolcache\windows\Python\3.11.0\x64\python.exe, platform=win32, version='3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]', encoding_fs_io=utf-8-cp1252) [virtualenv\discovery\builtin.py:65]
python: 343 D symlink on filesystem does work [virtualenv\info.py:43]
python: 343 D filesystem is not case-sensitive [virtualenv\info.py:24]
python: 390 W skip building and installing the package [tox\tox_env\runner.py:147]
  python: OK (0.06 seconds)
  congratulations :) (0.17 seconds)

Minimal example

I'm not sure, actually. But my suspects are --skip-pkg-install or --notest --installpkg ... — the former is followed by the latter, similar to tox's own CI setup.

The CI log is here: https://github.com/ansible-community/ansible-pygments/actions/runs/3680979306/jobs/6227176796#step:11:25.

[gaborbernat]

You're free to put in a PR to fix this, or at least a minimal reproducer.

[webknjaz]

So I used https://github.com/marketplace/actions/debugging-with-tmate to get into the CI env where it is misbehaving and discovered that my assumption about --skip-pkg-install was incorrect.

The culprit is running Python in isolated mode.

The minimum reproducer is python -Im tox + Windows.
It does not have any problems with finding the config on other platforms.
Also, python -m tox and tox are able to find the config on Windows too.

tox3 doesn't have a problem with python -Im tox on Windows either.

[webknjaz]

In isolated mode, https://github.com/tox-dev/tox/blob/32879c8/src/tox/config/source/discover.py#L37C9-L37C31 is None, while in non-isolated mode it's a ToxIni object. I wonder why.

[webknjaz]

Apparently, https://github.com/tox-dev/tox/blob/32879c8/src/tox/config/source/discover.py#L63C24-L63C43 throws a UnicodeDecodeError on my tox.ini. Likely, because there's an emoji inside.

[webknjaz]

The offending character is ❌: the exact position where decoding chokes is https://github.com/sphinx-contrib/sphinxcontrib-towncrier/blob/a93edfe/tox.ini#L122C15-L122C16.

[webknjaz]

So I checked what the interpreter does and apparently, it computes different the encoding depending on the presence of isolation… None of the important env vars are getting thrown away:

$ python -c 'import locale; print(locale.getpreferredencoding())'
utf-8

$ python -I -c 'import locale; print(locale.getpreferredencoding())'
cp1252

$ python -c 'import os; print("\n".join(sorted(dict(os.environ).keys())))' > env-vars.nonisolated

$ python -I -c 'import os; print("\n".join(sorted(dict(os.environ).keys())))' > env-vars.isolated

$ diff -u env-vars.nonisolated env-vars.isolated 

$ sha256sum env-vars.nonisolated env-vars.isolated
2aa38b9e955dfed93fe22e860910cdabfceacf82f95af54e951a7078c9db0a83 *env-vars.nonisolated
2aa38b9e955dfed93fe22e860910cdabfceacf82f95af54e951a7078c9db0a83 *env-vars.isolated

[webknjaz]

@gaborbernat any ideas on the right behavior? Do we declare this a CPython bug?

[webknjaz]

Also, how is it that tox3 works and tox4 doesn't? Did it use to have a workaround or something?

[webknjaz]

Executing chcp 65001 doesn't help Python notice the code page change...

[webknjaz]

This might be a workaround:

$ python -X utf8 -I -c 'import locale; print(locale.getpreferredencoding())'
utf-8

[webknjaz]

Yep, the workaround works:

# python -X utf8 -Im tox --notest -vvvvv -r
ROOT: 188 D setup logging to NOTSET on pid 1080 [tox\report.py:222]
folder=WindowsPath('D:/a/sphinxcontrib-towncrier/sphinxcontrib-towncrier')
base=WindowsPath('D:/a/sphinxcontrib-towncrier/sphinxcontrib-towncrier')
candidate=WindowsPath('D:/a/sphinxcontrib-towncrier/sphinxcontrib-towncrier/tox.ini') src_type=<class 'tox.config.source.tox_ini.ToxIni'> src_type.FILENAME='tox.ini'
src=ToxIni(path=D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\tox.ini)
source=ToxIni(path=D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\tox.ini) parsed.config_file=None parsed.root_dir=None
py: 928 D install pip from wheel C:\hostedtoolcache\windows\Python\3.13.2\x64\Lib\site-packages\virtualenv\seed\wheels\embed\pip-25.0.1-py3-none-any.whl via CopyPipInstall [virtualenv\seed\embed\via_app_data\via_app_data.py:51]
py: 929 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\py\Lib\site-packages\pip [virtualenv\util\path\_sync.py:42]
py: 1099 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip-25.0.1.dist-info to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\py\Lib\site-packages\pip-25.0.1.dist-info [virtualenv\util\path\_sync.py:42]
py: 1103 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip-25.0.1.virtualenv to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\py\Lib\site-packages\pip-25.0.1.virtualenv [virtualenv\util\path\_sync.py:42]
py: 1106 D generated console scripts pip-3.13.exe pip.exe pip3.13.exe pip3.exe [virtualenv\seed\embed\via_app_data\pip_install\base.py:45]
.pkg: 4347 D install pip from wheel C:\hostedtoolcache\windows\Python\3.13.2\x64\Lib\site-packages\virtualenv\seed\wheels\embed\pip-25.0.1-py3-none-any.whl via CopyPipInstall [virtualenv\seed\embed\via_app_data\via_app_data.py:51]
.pkg: 4349 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\.pkg\Lib\site-packages\pip [virtualenv\util\path\_sync.py:42]
.pkg: 4561 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip-25.0.1.dist-info to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\.pkg\Lib\site-packages\pip-25.0.1.dist-info [virtualenv\util\path\_sync.py:42]
.pkg: 4565 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.13\image\1\CopyPipInstall\pip-25.0.1-py3-none-any\pip-25.0.1.virtualenv to D:\a\sphinxcontrib-towncrier\sphinxcontrib-towncrier\.tox\.pkg\Lib\site-packages\pip-25.0.1.virtualenv [virtualenv\util\path\_sync.py:42]
.pkg: 4568 D generated console scripts pip-3.13.exe pip3.13.exe pip3.exe pip.exe [virtualenv\seed\embed\via_app_data\pip_install\base.py:45]
  py: OK (14.89 seconds)
  congratulations :) (15.00 seconds)

(a few odd lines at the beginning are print() calls I added while debugging)

[webknjaz]

Similarly to #3030, I think that there should be better logging @ https://github.com/tox-dev/tox/blob/32879c8/src/tox/config/source/discover.py#L65C17-L65C21 at least in debug mode. Although, it might be a good idea to separately handle a few known inherited exceptions and print out mitigation strategies...

[webknjaz]

xref: #1550 (tox3)