Updates to PEP 660 (GH-1978)

Author: sbidoul

pep-0660.rst

@@ -1,5 +1,5 @@
 PEP: 660
-Title: Editable installs for PEP 517 style build backends
+Title: Editable installs for pyproject.toml based builds
 Author: Daniel Holth <[email protected]>, Stéphane Bidoul <[email protected]>
 Sponsor: Paul Moore <[email protected]>
 Discussions-To: https://discuss.python.org/t/draft-pep-editable-installs-for-pep-517-style-build-backends/8510
@@ -67,11 +67,14 @@ require a compilation and/or installation step to become effective. The exact
 steps to perform will remain specific to the build backend used.
 
 When a project is installed in editable mode, users expect the installation to
-behave identically as a regular installation. Depending on the way build
-backends implement this specification, some minor differences may be visible
-such as the presence of additional files that are in the source tree and would
-not be part of a regular install. Build backends are encouraged to document
-such potential differences.
+behave identically as a regular installation. In particular the code must be
+importable by other code, and metadata must be available to standard mechanisms
+such as ``importlib.metadata``.
+
+Depending on the way build backends implement this specification, some minor
+differences may be visible such as the presence of additional files that are in
+the source tree and would not be part of a regular install. Build backends are
+encouraged to document such potential differences.
 
 The Mechanism
 =============
@@ -85,17 +88,9 @@ build_wheel_for_editable
 
 :: 
 
-  def build_wheel_for_editable(
-      wheel_directory,
-      scheme=scheme, 
-      config_settings=None):
+  def build_wheel_for_editable(wheel_directory, config_settings=None):
       ...
 
-``scheme``: a dictionary of installation categories ``{ 'purelib':
-'/home/py/.../site-packages', 'platlib': '...'}``. This makes it possible to
-use relative paths to the source code, which might help the interpreter find
-the package after the root path changes with ``chroot`` or similar.
-
 Must build a ``.whl`` file, and place it in the specified ``wheel_directory``.
 It must return the basename (not the full path) of the .whl file it creates, as
 a unicode string.
@@ -114,11 +109,11 @@ Build-backends must produce wheels that have the same dependencies
 with the exception that they can add dependencies necessary for their editable
 mechanism to function at runtime (such as `editables`_).
 
-The filename for the “editable” wheel needs to be PEP 427 compliant too. It
+The filename for the "editable" wheel needs to be PEP 427 compliant too. It
 does not need to use the same tags as ``build_wheel`` but it must be tagged as
 compatible with the system.
 
-An “editable” wheel uses the wheel format not for distribution but as ephemeral
+An "editable" wheel uses the wheel format not for distribution but as ephemeral
 communication between the build system and the front end. This avoids having
 the build backend install anything directly. This wheel must not be exposed
 to end users, nor cached, nor distributed.
@@ -142,7 +137,8 @@ If not defined, the default implementation is equivalent to ``return []``.
 What to put in the wheel
 ------------------------
 
-Build backends may use different techniques to achive the goals of an editable
+Build backends must populate the generated wheel with files that when installed will result in an editable install. 
+Build backends may use different techniques to achieve the goals of an editable
 install. This section provides examples and is not normative.
 
 * Build backends may choose to place a ``.pth`` file at the root of the ``.whl`` file,
@@ -157,11 +153,21 @@ install. This section provides examples and is not normative.
   a path importable, often including the project's own ``setup.py`` and other
   scripts that would not be part of a normal installation. The proxy strategy
   can achieve a higher level of fidelity than path-based methods.
+* Symbolic links are another useful mechanism to realize editable installs.
+  Since, at the time this writing, the ``wheel`` specification does not support
+  symbolic links, they are not directly usable to set-up symbolic links in the
+  target environment. It is however possible for the backend to create a
+  symlink structure in some ``build`` directory of the source tree, and add
+  that directory to the python path via a ``.pth`` file in the "editable"
+  wheel. If some files linked in this manner depend on python implementation or
+  version, ABI or platform, care must be taken to generate the link structure
+  in different directories depending on compatibility tags, so the same project
+  tree can be installed in editable mode in multiple environments.
 
 Frontend requirements
 ---------------------
 
-Frontends must install editable wheels in the same way as regular wheels.
+Frontends must install "editable" wheels in the same way as regular wheels.
 This also means uninstallation of editables does not require any special treatment.
 
 Frontends must create a ``direct_url.json`` file in the ``.dist-info``
@@ -186,15 +192,73 @@ Frontends must not expose the wheel obtained from ``build_wheel_for_editable``
 to end users. The wheel must be discarded after installation and must not be
 cached nor distributed.
 
+Limitations
+===========
+
+With regard to the wheel ``.data`` directory, this PEP focuses on making the
+``purelib`` and ``platlib`` categories (installed into site-packages)
+"editable". It does not make special provision for the other categories such as
+``headers``, ``data`` and ``scripts``. Package authors are encouraged to use
+``console_scripts``, make their ``scripts`` tiny wrappers around library
+functionality, or manage these from the source checkout during development.
+
 Rejected ideas
 ==============
 
+``editable`` local version identifier
+-------------------------------------
+
 The ideas of having build backends append or modify the local version
 identifier to include the ``editable`` string has been rejected because it
 would not satisfy ``==`` version speicifier that include the local version
-identifier. In other workds ``pkg==1.0+local`` is not satisfied by version
+identifier. In other words ``pkg==1.0+local`` is not satisfied by version
 ``1.0+local.editable``.
 
+Virtual wheel
+-------------
+
+Another approach was proposed in PEP 662, where
+the build backend returns a mapping from source files and directories to the
+installed layout. It is then up to the installer frontend to realize the
+editable installation by whatever means it deems adequate for its users.
+
+In terms of capabilities, both proposals provide the core "editable" feature.
+
+The key difference is that PEP 662 leaves it to the frontend to decide how the
+editable installation will be realized, while with this PEP, the choice must be
+made by the backend. Both approaches can in principle provide several editable
+installation methods for a given project, and let the developer choose one at
+install time.
+
+At the time of writing this PEP, it is clear that the community has a wide
+range of theoretical and practical expectations about editable installs. The
+reality is that the only one there is wide experience with is path insertion
+via .pth (i.e. what setup.py develop does).
+
+We believe that PEP 660 better addresses these "unknown unknowns" today in the
+most reliable way, by letting project authors select the backend or implement
+the method that provides the editable mechanism that best suit their
+requirements, and test it works correctly. Since the frontend has no latitude
+in *how* to install the "editable" wheel, in case of issue, there is only one
+place to investigate: the build backend.
+
+With PEP 662, issues need to be investigated in the frontend,
+the backend and possiblty the specification. There is also a high probability
+that different frontends, implementing the specification in different ways,
+will produce installations that behave differently than project authors
+intended, creating confusion, or worse, projects that only work with specific
+frontends or IDEs.
+
+Unpacked wheel
+--------------
+
+A `prototype <https://github.com/pypa/pip/pull/8154/files>`_ was made that
+created an unpacked wheel in a temporary directory, to be copied to the target
+environment by the frontend. This approach was not pursued because a wheel
+archive is easy to create for the backend, and using a wheel as communication
+mechanism is a better fit with the PEP 517 philosophy, and therefore keeps
+things simpler for the frontend.
+
 References
 ==========