Skip to content

Virtualenv corrupts dist-info during embedded downgrade overlaying existing venv #2185

@arcivanov

Description

@arcivanov

Scenario:

  1. Virtualenv carries outdated embedded setuptools (or any other internal essential).
  2. Rollout virtualenv.
  3. Activate rolled out venv.
  4. Upgrade setuptools with pip.
  5. Deactivate venv.
  6. Rollout virtualenv again on top of the existing venv.
  7. Observe multiple dist-info directories present for incorrectly downgraded tool, resulting in venv corruption.

Example - always reproducible:

(pyb-3.9.7) [username@hostname pybuilder]$ pip install virtualenv\<20.8
Collecting virtualenv<20.8
  Using cached virtualenv-20.7.2-py2.py3-none-any.whl (5.3 MB)
Requirement already satisfied: distlib<1,>=0.3.1 in /home/username/.pyenv/versions/3.9.7/envs/pyb-3.9.7/lib/python3.9/site-packages (from virtualenv<20.8) (0.3.2)
Requirement already satisfied: backports.entry-points-selectable>=1.0.4 in /home/username/.pyenv/versions/3.9.7/envs/pyb-3.9.7/lib/python3.9/site-packages (from virtualenv<20.8) (1.1.0)
Requirement already satisfied: platformdirs<3,>=2 in /home/username/.pyenv/versions/3.9.7/envs/pyb-3.9.7/lib/python3.9/site-packages (from virtualenv<20.8) (2.3.0)
Requirement already satisfied: six<2,>=1.9.0 in /home/username/.pyenv/versions/3.9.7/envs/pyb-3.9.7/lib/python3.9/site-packages (from virtualenv<20.8) (1.16.0)
Requirement already satisfied: filelock<4,>=3.0.0 in /home/username/.pyenv/versions/3.9.7/envs/pyb-3.9.7/lib/python3.9/site-packages (from virtualenv<20.8) (3.0.12)
Installing collected packages: virtualenv
  Attempting uninstall: virtualenv
    Found existing installation: virtualenv 20.8.0
    Uninstalling virtualenv-20.8.0:
      Successfully uninstalled virtualenv-20.8.0
Successfully installed virtualenv-20.7.2
(pyb-3.9.7) [username@hostname pybuilder]$ virtualenv test-env
created virtual environment CPython3.9.7.final.0-64 in 113ms
  creator CPython3Posix(dest=/home/username/src/pybuilder/test-env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/username/.local/share/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
(pyb-3.9.7) [username@hostname pybuilder]$ source test-env/bin/activate
(test-env) (pyb-3.9.7) [username@hostname pybuilder]$ pip install -U setuptools
Requirement already satisfied: setuptools in ./test-env/lib/python3.9/site-packages (57.4.0)
Collecting setuptools
  Using cached setuptools-58.0.4-py3-none-any.whl (816 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 57.4.0
    Uninstalling setuptools-57.4.0:
      Successfully uninstalled setuptools-57.4.0
Successfully installed setuptools-58.0.4
WARNING: You are using pip version 21.2.3; however, version 21.2.4 is available.
You should consider upgrading via the '/home/username/src/pybuilder/test-env/bin/python -m pip install --upgrade pip' command.
(test-env) (pyb-3.9.7) [username@hostname pybuilder]$ deactivate
(pyb-3.9.7) [username@hostname pybuilder]$ ls -la test-env/lib/python3.9/site-packages/
total 24
drwxr-xr-x 11 username username 4096 Sep 16 13:34 .
drwxr-xr-x  3 username username   27 Sep 16 13:34 ..
drwxr-xr-x  3 username username   63 Sep 16 13:34 _distutils_hack
-rw-r--r--  1 username username  152 Sep 16 13:34 distutils-precedence.pth
drwxr-xr-x  5 username username  111 Sep 16 13:34 pip
drwxr-xr-x  2 username username  130 Sep 16 13:34 pip-21.2.3.dist-info
-rw-r--r--  1 username username    0 Sep 16 13:34 pip-21.2.3.virtualenv
drwxr-xr-x  6 username username   86 Sep 16 13:34 pkg_resources
drwxr-xr-x  2 username username   40 Sep 16 13:34 __pycache__
drwxr-xr-x  7 username username 4096 Sep 16 13:34 setuptools
drwxr-xr-x  2 username username  143 Sep 16 13:34 setuptools-58.0.4.dist-info
-rw-r--r--  1 username username   18 Sep 16 13:34 _virtualenv.pth
-rw-r--r--  1 username username 5662 Sep 16 13:34 _virtualenv.py
drwxr-xr-x  4 username username  190 Sep 16 13:34 wheel
drwxr-xr-x  2 username username  130 Sep 16 13:34 wheel-0.37.0.dist-info
-rw-r--r--  1 username username    0 Sep 16 13:34 wheel-0.37.0.virtualenv
(pyb-3.9.7) [username@hostname pybuilder]$ virtualenv test-env
created virtual environment CPython3.9.7.final.0-64 in 128ms
  creator CPython3Posix(dest=/home/username/src/pybuilder/test-env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/username/.local/share/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, setuptools==58.0.4, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
(pyb-3.9.7) [username@hostname pybuilder]$ ls -la test-env/lib/python3.9/site-packages/
total 24
drwxr-xr-x 12 username username 4096 Sep 16 13:35 .
drwxr-xr-x  3 username username   27 Sep 16 13:34 ..
drwxr-xr-x  2 username username   44 Sep 16 13:35 _distutils_hack
-rw-r--r--  1 username username  152 Sep 16 13:35 distutils-precedence.pth
drwxr-xr-x  4 username username   92 Sep 16 13:35 pip
drwxr-xr-x  2 username username  130 Sep 16 13:35 pip-21.2.3.dist-info
-rw-r--r--  1 username username    0 Sep 16 13:35 pip-21.2.3.virtualenv
drwxr-xr-x  5 username username   67 Sep 16 13:35 pkg_resources
drwxr-xr-x  2 username username   40 Sep 16 13:34 __pycache__

********  <=======================
drwxr-xr-x  6 username username 4096 Sep 16 13:35 setuptools
drwxr-xr-x  2 username username  126 Sep 16 13:35 setuptools-57.4.0.dist-info
-rw-r--r--  1 username username    0 Sep 16 13:35 setuptools-57.4.0.virtualenv
drwxr-xr-x  2 username username  143 Sep 16 13:34 setuptools-58.0.4.dist-info
*********

-rw-r--r--  1 username username   18 Sep 16 13:35 _virtualenv.pth
-rw-r--r--  1 username username 5662 Sep 16 13:35 _virtualenv.py
drwxr-xr-x  4 username username  190 Sep 16 13:35 wheel
drwxr-xr-x  2 username username  130 Sep 16 13:35 wheel-0.37.0.dist-info
-rw-r--r--  1 username username    0 Sep 16 13:35 wheel-0.37.0.virtualenv

This environment is now broken producing pybuilder/pybuilder#795 due to setuptools-57.4.0.dist-info and setuptools-58.0.4.dist-info being present.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions