WheelDistribution doesn't implement abstract methods of importlib.metadata.Distribution

[jaraco]

Description

In python/cpython#100466, we learned that pip implements a WheelDistribution derived from importlib.metadata.Distribution, but doesn't implement the abstract methods. This usage was allowed because the ABC didn't derive from abc.ABCMeta. I don't recall if that choice was intentional or not, but attempting to enforce the definition of abstract methods in subclasses breaks pip invocations.

Expected behavior

Pip should implement the abstract methods (even if just trivially).

pip version

22.3.1

Python version

3.12a3ish

OS

all

How to Reproduce

Apply the patch from the referenced Python PR to the Python stdlib and invoke pip install.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[ichard26]

It should be noted that partial implementation of the Distribution interface is now deprecated: python/importlib_metadata#451

[jaraco]

Now that the behavior is in Python 3.12, the repro is:

 draft 🐚 py -m venv .venv
 draft 🐚 py -W error::DeprecationWarning:pip._internal.metadata.importlib._dists -m pip install sampleproject
Collecting sampleproject
  Using cached sampleproject-3.0.0-py3-none-any.whl.metadata (4.4 kB)
Collecting peppercorn (from sampleproject)
  Using cached peppercorn-0.6-py3-none-any.whl.metadata (3.4 kB)
Using cached sampleproject-3.0.0-py3-none-any.whl (4.7 kB)
Using cached peppercorn-0.6-py3-none-any.whl (4.8 kB)
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/jaraco/.local/pip-run/pip/_internal/cli/base_command.py", line 179, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/jaraco/.local/pip-run/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/Users/jaraco/.local/pip-run/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
        reqs, check_supported_wheels=not options.target_dir
    )
  File "/Users/jaraco/.local/pip-run/pip/_internal/resolution/resolvelib/resolver.py", line 179, in resolve
    self.factory.preparer.prepare_linked_requirements_more(reqs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/jaraco/.local/pip-run/pip/_internal/operations/prepare.py", line 554, in prepare_linked_requirements_more
    self._complete_partial_requirements(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        partially_downloaded_reqs,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        parallel_builds=parallel_builds,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/jaraco/.local/pip-run/pip/_internal/operations/prepare.py", line 489, in _complete_partial_requirements
    self._prepare_linked_requirement(req, parallel_builds)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/.local/pip-run/pip/_internal/operations/prepare.py", line 642, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
        req,
    ...<3 lines>...
        self.check_build_deps,
    )
  File "/Users/jaraco/.local/pip-run/pip/_internal/operations/prepare.py", line 75, in _get_prepared_distribution
    return abstract_dist.get_metadata_distribution()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/jaraco/.local/pip-run/pip/_internal/distributions/wheel.py", line 34, in get_metadata_distribution
    return get_wheel_distribution(wheel, canonicalize_name(self.req.name))
  File "/Users/jaraco/.local/pip-run/pip/_internal/metadata/__init__.py", line 106, in get_wheel_distribution
    return select_backend().Distribution.from_wheel(wheel, canonical_name)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/.local/pip-run/pip/_internal/metadata/importlib/_dists.py", line 133, in from_wheel
    dist = WheelDistribution.from_zipfile(zf, name, wheel.location)
  File "/Users/jaraco/.local/pip-run/pip/_internal/metadata/importlib/_dists.py", line 73, in from_zipfile
    return cls(files, info_location)
  File "/opt/python/lib/python3.13/importlib/metadata/__init__.py", line 344, in __new__
    warnings.warn(
    ~~~~~~~~~~~~~^
        f"Unimplemented abstract methods {abstract}",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        DeprecationWarning,
        ^^^^^^^^^^^^^^^^^^^
        stacklevel=2,
        ^^^^^^^^^^^^^
    )
    ^
DeprecationWarning: Unimplemented abstract methods {'locate_file'}

And we can probably expect pip to start failing in Python 3.14.

[jaraco]

So I've taken a look at the code. I see now that pip implements the WheelDistribution because importlib.metadata.PathDistribution holds the zipfile.Path open too long for pip's needs.

I wonder if a better way to deal with the situation is to eliminate this concern (and the need for WheelDistribution altogether). Did you know that zipfile.Path can open a zipfile in memory? How much pain would users notice if pip were to load the whole wheel into memory for the life of the Distribution object? If that's too much overhead, one could simply construct a metadata-only wheel (in-memory) and pass that to PathDistribution, effectively implementing the same behavior that's there currently, but without creating a whole new WheelDistribution object.

[pfmoore]

I'm going to push back on this because you (or more accurately the importlib.metadata docs) haven't explained why our implementation isn't acceptable. It may well be that we could do as you say, but that's new, fragile, code that would need writing and maintaining, and surely the whole point of allowing user implementations of distribution objects is so that users can implement custom behaviours - if the docs don't explain the limitations on doing so, the docs should be fixed first, so that users can be sure that any reimplementation required of them will be supported going forward, and won't be subject to future unexpected breakages like this one.