pip 23.0.1 and anyio==4.0.0 break installation with --upgrade --no-binary :all: options

[i-chvets]

Description

When trying to install dependency anyio, setuptools_scm installation is broken when using options --upgrade --no-binary :all:
pip version is 23.0.1
Error observed:

LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz

Steps to reproduce

pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
  Using cached anyio-4.0.0.tar.gz (153 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

Full log

$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting anyio==4.0.0
  Using cached anyio-4.0.0.tar.gz (153 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [116 lines of output]
      DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
      Collecting setuptools>=64
        Using cached setuptools-68.2.2.tar.gz (2.2 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools_scm>=6.4
        Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'error'
        error: subprocess-exited-with-error
      
        × pip subprocess to install build dependencies did not run successfully.
        │ exit code: 1
        ╰─> [85 lines of output]
            DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
            Ignoring typing_extensions: markers 'python_version < "3.8"' don't match your environment
            Collecting importlib-metadata>=4.6
              Using cached importlib_metadata-6.8.0.tar.gz (53 kB)
              Installing build dependencies: started
              Installing build dependencies: finished with status 'error'
              error: subprocess-exited-with-error
      
              × pip subprocess to install build dependencies did not run successfully.
              │ exit code: 2
              ╰─> [61 lines of output]
                  DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
                  Collecting setuptools>=56
                    Using cached setuptools-68.2.2.tar.gz (2.2 MB)
                    Getting requirements to build wheel: started
                    Getting requirements to build wheel: finished with status 'done'
                    Installing backend dependencies: started
                    Installing backend dependencies: finished with status 'done'
                    Preparing metadata (pyproject.toml): started
                    Preparing metadata (pyproject.toml): finished with status 'done'
                  Collecting setuptools_scm[toml]>=3.4.1
                    Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
                  ERROR: Exception:
                  Traceback (most recent call last):
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
                      status = run_func(*args)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
                      return func(self, options, args)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 419, in run
                      requirement_set = resolver.resolve(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
                      result = self._result = resolver.resolve(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
                      state = resolution.resolve(requirements, max_rounds=max_rounds)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
                      self._add_to_criteria(self.state.criteria, r, parent=None)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
                      if not criterion.candidates:
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
                      return bool(self._sequence)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
                      return any(self)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
                      return (c for c in iterator if id(c) not in self._incompatible_ids)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
                      candidate = func()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
                      self._link_candidate_cache[link] = LinkCandidate(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
                      super().__init__(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
                      self.dist = self._prepare()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
                      dist = self._prepare_distribution()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
                      return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
                      return self._prepare_linked_requirement(req, parallel_builds)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
                      dist = _get_prepared_distribution(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
                      with build_tracker.track(req):
                    File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
                      return next(self.gen)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 122, in track
                      self.add(req)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 92, in add
                      raise LookupError(message)
                  LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz
      
                  [notice] A new release of pip is available: 23.0.1 -> 23.2.1
                  [notice] To update, run: pip install --upgrade pip
                  [end of output]
      
              note: This error originates from a subprocess, and is likely not a problem with pip.
            error: subprocess-exited-with-error
      
            × pip subprocess to install build dependencies did not run successfully.
            │ exit code: 2
            ╰─> See above for output.
      
            note: This error originates from a subprocess, and is likely not a problem with pip.
      
            [notice] A new release of pip is available: 23.0.1 -> 23.2.1
            [notice] To update, run: pip install --upgrade pip
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error
      
      × pip subprocess to install build dependencies did not run successfully.
      │ exit code: 1
      ╰─> See above for output.
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      
      [notice] A new release of pip is available: 23.0.1 -> 23.2.1
      [notice] To update, run: pip install --upgrade pip
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behaviour

setuptools_scm is installed with no error messages.

Workaround

Not available.

[RonnyPfannschmidt]

this cycle was previously missed, im not quite sure how to unravel

[RonnyPfannschmidt]

packaging will be removed as build dependency and i'll use the setuptools vendored one in that case

[i-chvets]

Just verified and it still fails with 8.0.2 release.Can we re-open issue to track this?

Full log of test:

$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
  Downloading anyio-4.0.0.tar.gz (153 kB)
     |████████████████████████████████| 153 kB 4.0 MB/s 
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4'
       cwd: None
  Complete output (62 lines):
  Collecting setuptools>=64
    Downloading setuptools-68.2.2.tar.gz (2.2 MB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting setuptools_scm>=6.4
    Downloading setuptools-scm-8.0.2.tar.gz (72 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"'
         cwd: None
    Complete output (44 lines):
    Ignoring typing-extensions: markers 'python_version < "3.8"' don't match your environment
    Collecting importlib-metadata>=4.6
      Downloading importlib_metadata-6.8.0.tar.gz (53 kB)
      Installing build dependencies: started
      Installing build dependencies: finished with status 'error'
      ERROR: Command errored out with exit status 2:
       command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1'
           cwd: None
      Complete output (33 lines):
      Collecting setuptools>=56
        Using cached setuptools-68.2.2.tar.gz (2.2 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
          Preparing wheel metadata: started
          Preparing wheel metadata: finished with status 'done'
      Collecting setuptools_scm[toml]>=3.4.1
        Using cached setuptools-scm-8.0.2.tar.gz (72 kB)
      ERROR: Exception:
      Traceback (most recent call last):
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
          status = self.run(options, args)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 357, in run
          resolver.resolve(requirement_set)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
          discovered_reqs.extend(self._resolve_one(requirement_set, req))
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
          abstract_dist = self._get_abstract_dist_for(req_to_install)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
          abstract_dist = self.preparer.prepare_linked_requirement(req)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
          abstract_dist = _get_prepared_distribution(
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
          with req_tracker.track(req):
        File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
          return next(self.gen)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
          self.add(req)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
          raise LookupError(message)
      LookupError: https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005 (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005
      ----------------------------------------
    ERROR: Command errored out with exit status 2: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1' Check the logs for full command output.
    ----------------------------------------
  ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"' Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4' Check the logs for full command output.

[RonnyPfannschmidt]

 podman run -it --entrypoint /bin/bash python  "-c" "pip3 install --upgrade --no-binary :all: anyio==4.0.0"
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting anyio==4.0.0
  Downloading anyio-4.0.0.tar.gz (153 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.3/153.3 kB 3.9 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting idna>=2.8
  Downloading idna-3.4.tar.gz (183 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 183.1/183.1 kB 9.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting sniffio>=1.1
  Downloading sniffio-1.3.0.tar.gz (17 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: anyio, idna, sniffio
  Building wheel for anyio (pyproject.toml) ... done
  Created wheel for anyio: filename=anyio-4.0.0-py3-none-any.whl size=83122 sha256=14d6a7627545464b0f19e8b7ff6a1a87fbba475785eee35e7d134bcab8858cf6
  Stored in directory: /root/.cache/pip/wheels/f5/23/5a/b1dfb39b8a1fef8c850b021366051c9b8017b320a185faa7a1
  Building wheel for idna (pyproject.toml) ... done
  Created wheel for idna: filename=idna-3.4-py3-none-any.whl size=61538 sha256=eaf916f74ea1a0d0a5a2de80ad62503d1a42d2e7a413e512808add7765e645e6
  Stored in directory: /root/.cache/pip/wheels/73/14/5f/87e450624ca0598cd1347aaac250ba069435fdc821c99b0e3b
  Building wheel for sniffio (pyproject.toml) ... done
  Created wheel for sniffio: filename=sniffio-1.3.0-py3-none-any.whl size=10166 sha256=12bf3c7f732ddbcb94fed28d717a5c1920fd7050a3bdda1aae2153a406830325
  Stored in directory: /root/.cache/pip/wheels/d5/1c/0a/a5c736dd17400583869e23f40145928205a7a57d7304c491df
Successfully built anyio idna sniffio
Installing collected packages: sniffio, idna, anyio
Successfully installed anyio-4.0.0 idna-3.4 sniffio-1.3.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

seems to hit pypa/pip#11453

[RonnyPfannschmidt]

@i-chvets so the current recommendation is a pip upgrade it seems

[DnPlas]

Hey @RonnyPfannschmidt, I tried upgrading pip, but I still cannot manage to solve the issue:

(venv) ubuntu@charm-dev-focal:~$ pip --version
pip 23.2.1 from /home/ubuntu/venv/lib/python3.8/site-packages/pip (python 3.8)

(venv) ubuntu@charm-dev-focal:~$ pip3 install --upgrade --no-binary :all: setuptools-scm
...
LookupError: https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz
                  [end of output]