install-pdm.py error: No pyvenv.cfg file

[eight04]

• I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

1, Download https://pdm.fming.dev/install-pdm.py
2. Run python install-pdm.py

Actual behavior

C:\Users\eight04\Downloads>python install-pdm.py
Installing PDM (2.9.3): Creating virtual environment
Actual environment location may have moved due to redirects, links or junctions.
  Requested location: "C:\Users\eight04\AppData\Roaming\pdm\venv\Scripts\python.exe"
  Actual location:    "C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\Roaming\pdm\venv\Scripts\python.exe"
Installing PDM (2.9.3): Installing PDM and dependencies
An error occurred when executing ['C:\\Users\\eight04\\AppData\\Roaming\\pdm\\venv\\Scripts\\python.exe', '-m', 'ensurepip']:
No pyvenv.cfg file

An error occurred when executing ['C:\\Users\\eight04\\AppData\\Roaming\\pdm\\venv\\Scripts\\python.exe', '-m', 'pip', 'install', '-IU', 'pip']:
No pyvenv.cfg file

Expected behavior

Install pdm

Environment Information

Windows 10
Python 3.10 from MS Store

[fromm1990]

I have the same issue using Python 3.12 from MS Store on Windows 11.
Installing PDM using binaries installed from python.org works fine.

[frostming]

Does the problematic python has venv module?

[eight04]

I upgraded to python 3.12 and can still reproduce.

Does the problematic python has venv module?

I think so?

C:\Users\eight04\Downloads>python -c "import venv;print(venv)"
<module 'venv' from 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1264.0_x64__qbz5n2kfra8p0\\Lib\\venv\\__init__.py'>

[frostming]

According to the error message it reported:

No pyvenv.cfg file

Does it exist in any venv created by python -m venv? Can venv/bin/python -m ensurepip execute successfully?

[eight04]

I looked into C:\Users\eight04\AppData\Roaming and there is no pdm folder. Does install-pdm remove the directory on failure?

Does it exist in any venv created by python -m venv? Can venv/bin/python -m ensurepip execute successfully?

Everything works fine:

C:\Users\eight04>python -m venv venv_test

C:\Users\eight04>type venv_test\pyvenv.cfg
home = C:\Users\eight04\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0
include-system-site-packages = false
version = 3.12.4
executable = C:\Users\eight04\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\python.exe
command = C:\Users\eight04\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\python.exe -m venv C:\Users\eight04\venv_test

C:\Users\eight04>venv_test\Scripts\python.exe -m ensurepip
Looking in links: c:\Users\eight04\AppData\Local\Temp\tmp_ooi8w6t
Requirement already satisfied: pip in c:\users\eight04\venv_test\lib\site-packages (24.0)

[frostming]

I looked into C:\Users\eight04\AppData\Roaming and there is no pdm folder. Does install-pdm remove the directory on failure?

Nope, I suspect the venv is not created successfully. Can you help dig into this further since I don't work on Windows.

Actual environment location may have moved due to redirects, links or junctions.

This line is also suspicious. The Python installed from microsoft store may be special.

[eight04]

So the venv is actually created at:

C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Roaming\pdm\venv\

Instead of:

C:\Users\eight04\AppData\Roaming\pdm\venv\

It seems that Python from MSStore has some special redirects:
https://docs.python.org/3/using/windows.html#redirection-of-local-data-registry-and-temporary-paths

At runtime, Python will use a private copy of well-known Windows folders and the registry. For example, if the environment variable %APPDATA% is c:\Users<user>\AppData, then when writing to C:\Users<user>\AppData\Local will write to C:\Users<user>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\Local.

And the redirect only happens when the path goes through python API:

from pathlib import Path
p = Path("C:\\Users\\eight04\\AppData\\Roaming\\pdm\\venv\\Scripts\\python.exe")
print(p.stat()) # ok

import os
os.system(f"{p} --version") # system cannot find the path

import subprocess
subprocess.run([p, "--version"]) # No pyvenv.cfg error

subprocess.run([p, "--version"], shell=True) # system cannot find the path

To execute python correctly, we have to call its real path:

subprocess.run([p.resolve(), "--version"]) # 3.12.4

Maybe relate:
python/cpython#86179

[frostming]

To execute python correctly, we have to call its real path:

subprocess.run([p.resolve(), "--version"]) # 3.12.4

Great, thanks for the investigation. So does it solve the problem if you add .resolve() to these lines?

pdm/install-pdm.py

Lines 280 to 283 in 81d8ec3

	if WINDOWS:
	venv_python = venv_path / "Scripts/python.exe"
	else:
	venv_python = venv_path / "bin/python"

If so, can you submit a PR?

[eight04]

Yep. I'll send a PR later.

C:\Users\eight04\Downloads>python install-pdm.py
Installing PDM (2.16.1): Creating virtual environment
Actual environment location may have moved due to redirects, links or junctions.
  Requested location: "C:\Users\eight04\AppData\Roaming\pdm\venv\Scripts\python.exe"
  Actual location:    "C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Roaming\pdm\venv\Scripts\python.exe"
Installing PDM (2.16.1): Installing PDM and dependencies
Installing PDM (2.16.1): Making binary at C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Scripts
Usage: pdm [-h] [-V] [-c CONFIG] [-v | -q] [--no-cache] [-I] [--pep582 [SHELL]] ...

    ____  ____  __  ___
   / __ \/ __ \/  |/  /
  / /_/ / / / / /|_/ /
 / ____/ /_/ / /  / /
/_/   /_____/_/  /_/

Options:
  -h, --help            Show this help message and exit.
  -V, --version         Show the version and exit
  -c CONFIG, --config CONFIG
                        Specify another config file path [env var: PDM_CONFIG_FILE]
  -v, --verbose         Use `-v` for detailed output and `-vv` for more detailed
  -q, --quiet           Suppress output
  --no-cache            Disable the cache for the current command. [env var: PDM_NO_CACHE]
  -I, --ignore-python   Ignore the Python path saved in .pdm-python. [env var: PDM_IGNORE_SAVED_PYTHON]
  --pep582 [SHELL]      Print the command line to be eval'd by the shell

Commands:
  add                   Add package(s) to pyproject.toml and install them
  build                 Build artifacts for distribution
  cache                 Control the caches of PDM
  completion            Generate completion scripts for the given shell
  config                Display the current configuration
  export                Export the locked packages set to other formats
  fix                   Fix the project problems according to the latest version of PDM
  import                Import project metadata from other formats
  info                  Show the project information
  init                  Initialize a pyproject.toml for PDM
  install               Install dependencies from lock file
  list                  List packages installed in the current working set
  lock                  Resolve and lock dependencies
  outdated              Check for outdated packages and list the latest versions on indexes.
  publish               Build and publish the project to PyPI
  python (py)           Manage installed Python interpreters
  remove                Remove packages from pyproject.toml
  run                   Run commands or scripts with local packages loaded
  search                Search for PyPI packages
  self (plugin)         Manage the PDM program itself (previously known as plugin)
  show                  Show the package information
  sync                  Synchronize the current working set with lock file
  update                Update package(s) in pyproject.toml
  use                   Use the given python version or path as base interpreter. If not found, PDM will try to
                        install one.
  venv                  Virtualenv management

Successfully installed: PDM (2.16.1) at C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Scripts\pdm.exe
Post-install: c:\users\eight04\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\scripts is added to PATH env, please restart your terminal to take effect

C:\Users\eight04\Downloads>C:\Users\eight04\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Scripts\pdm.exe --version
PDM, version 2.16.1