Show a better error message when PYTHONHOME is wrong

[konstin]

Feature or enhancement

Proposal:

Currently, when PYTHONHOME is wrong, python says ModuleNotFoundError: No module named 'encodings'. This is error makes it sound like a module is missing, when it's actually a part of the core interpreter configuration that is broken, which the error doesn't tell.

Ideally, the error would show if a pyvenv.cfg was loaded, as a wrong home path causes this error.

Example pyenv Python 3.12 on Ubuntu 24.04:

$ PYTHONHOME=/dev/null python
Python path configuration:
  PYTHONHOME = '/dev/null'
  PYTHONPATH = (not set)
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/dev/null/lib/python3.12'
  sys._base_executable = '/home/konsti/.local/bin/python'
  sys.base_prefix = '/dev/null'
  sys.base_exec_prefix = '/dev/null'
  sys.platlibdir = 'lib'
  sys.executable = '/home/konsti/.local/bin/python'
  sys.prefix = '/dev/null'
  sys.exec_prefix = '/dev/null'
  sys.path = [
    '/dev/null/lib/python312.zip',
    '/dev/null/lib/python3.12',
    '/dev/null/lib/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007a7eee702b80 (most recent call first):
  <no Python frame>

Example python-build-standalone on Ubuntu 24.04:

$ PYTHONHOME=/dev/null python3.13
Fatal Python error: Failed to import encodings module
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000072ef39049b80 (most recent call first):
  <no Python frame>

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

There are several bug reports featuring this error: https://github.com/python/cpython/issues?q=is%3Aissue+%22ModuleNotFoundError%3A+No+module+named+%27encodings%27%22+

https://stackoverflow.com/questions/38132755/importerror-no-module-named-encodings

The error is occurring for users in various communities: https://www.google.com/search?q=%22ModuleNotFoundError%3A+No+module+named+%27encodings%27%22

Linked PRs

• gh-125873: Improve error message when PYTHONHOME is invalid #126027
• GH-125873: error out in getpath when unable to find stdlib #127616

[ZeroIntensity]

I think the most reasonable fix would be to change this line to something like:

return PyStatus_Error("Failed to import encodings module. Are you sure PYTHONHOME is correct?");

PR welcome!

[skirpichev]

You can also damage PYTHONPLATLIBDIR, maybe more environment variables, e.g.:

$ PYTHONPLATLIBDIR=/dev/null python3.13 
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Fatal Python error: Failed to import encodings module
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fa577a2c740 (most recent call first):
  <no Python frame>

[ZeroIntensity]

You could, but what makes PYTHONHOME special is that it's "required" to do anything with the interpreter--explicitly messing with other variables is the user's fault IMO.

[FFY00]

Tangentially related: we should also fail earlier if PYTHONHOME, PYTHONPLATLIBDIR, etc. are not directories in the fist place.

[FFY00]

Related to GH-125873, where this failure mode is triggered by specifying a bad home in pyvenv.cfg. A custom error message should be used for that case, though.