Skip to content

PEP 751: A file format to list Python dependencies for installation reproducibility #3870

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 59 commits into from
Jul 24, 2024
Merged

PEP 751: A file format to list Python dependencies for installation reproducibility #3870

merged 59 commits into from
Jul 24, 2024

Conversation

brettcannon
Copy link
Member

@brettcannon brettcannon commented Jul 24, 2024 •
edited by hugovk
Loading

Basic requirements (all PEP Types)

  • Read and followed PEP 1 & PEP 12
  • File created from the latest PEP template
  • PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
  • Title clearly, accurately and concisely describes the content in 79 characters or less
  • Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
  • Author, Status (Draft), Type and Created headers filled out correctly
  • PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
  • Required sections included
    • Abstract (first section)
    • Copyright (last section; exact wording from template required)
  • Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
  • PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
  • Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

  • PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
  • Suggested sections included (unless not applicable)
    • Motivation
    • Rationale
    • Specification
    • Backwards Compatibility
    • Security Implications
    • How to Teach This
    • Reference Implementation
    • Rejected Ideas
    • Open Issues
  • Python-Version set to valid (pre-beta) future Python version, if relevant
  • Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
  • Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

📚 Documentation preview 📚: https://pep-previews--3870.org.readthedocs.build/pep-0751/

@brettcannon
Write up to the specification
5e9e274
@brettcannon
Tweak rationale
f4fc728
@brettcannon
Write down the file format
68a6dcd
@brettcannon
Add in the [[tool]] table array
761922e
@brettcannon
Add expectations for lockers
61a72f5
@brettcannon
Add expectations for installers for per-file locking
86e44fa
@brettcannon
Add some clarifications
a960f76
@brettcannon
Example workflow for package locking
08f59d3
@brettcannon
Bit of a clarification
82393b2
@brettcannon
Add package.tool
8c6801d
@brettcannon
Add package.dependents
6b188ba
@brettcannon
Fix a typo
6cced99
@brettcannon
Add package.description
0080fc2 
Along the way also add more explanation as to why some things are included
@brettcannon
Add package.direct, package.directory, and `package.multiple-entr…
1214a78 
…ies`
@brettcannon
Rename package.simple-repo-package-url and add `package.files.simpl…
92de818 
…e-repo-package-url`

Also tweak Core Metadata 2.4 to be simpler by outright banning varying metadata
@brettcannon
Updates based on some review comments
619bd6e 
- Drop core metadata 2.4
- Suggest SHA-256 or better
- Suggest lockers set the simple Repo package URL
- Rearrange the order of some keys
@brettcannon
Use a fake PEP number to make CI happy
244957c
@brettcannon
Add an explicit note that the directory to write a pylock.toml file…
6a92b8f 
… is not specified on purpose.
@brettcannon
Make the hash algorithm its own setting
2eb7166
@brettcannon
Fill out the ACKS section
3623ccf
@brettcannon
Outline rejected ideas
22fc531
@brettcannon
Add a rejected idea
3182912
@brettcannon
Record the rejected idea of new core metadata version for metadata co…
0e1df0c 
…nsistency
@brettcannon
Answer the rejected idea of having the installer do resolution
48811ee
@brettcannon
Fill in rejected ideas around file names
b1c2fdc
@brettcannon
Add rejected ideas around the file format
135a13e
@brettcannon
Finish filling in the rejected ideas
450a50f
@brettcannon
Minor tweaks
aa7840f
@brettcannon
Finish proofreading
f6b02ef
@brettcannon
Clarify that installers should never guess which [[file-lock]] to i…
b35057a 
…nstall
AA-Turner
AA-Turner reviewed Jul 24, 2024
View reviewed changes
Copy link
Member

@AA-Turner AA-Turner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A brief editorial review (I havent't looked at rejected ideas in great detail):

@AA-Turner AA-Turner added the new-pep A new draft PEP submitted for initial review label Jul 24, 2024
hugovk
hugovk reviewed Jul 24, 2024
View reviewed changes
JelleZijlstra
JelleZijlstra reviewed Jul 24, 2024
View reviewed changes
willingc
willingc reviewed Jul 24, 2024
View reviewed changes
brettcannon and others added 14 commits July 24, 2024 13:29
@brettcannon
Accept/address some comments
1b39e49
@brettcannon @hugovk
@AA-Turner @JelleZijlstra
Apply suggestions from code review
9031e5c 
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
@brettcannon @JelleZijlstra @hugovk
Apply suggestions from code review
fd139c3 
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
@brettcannon @hugovk
@JelleZijlstra @AA-Turner
Apply suggestions from code review
89ff321 
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
@brettcannon @JelleZijlstra @hugovk
Apply suggestions from code review
a914c12 
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
@brettcannon
Adam likes periods
8c21909
@brettcannon
More details around hashing
98b4e82
@brettcannon
Clarify package.direct defaults to false
e27b73d 
There were some questions around whether this being optional meant there was an actual tri-state, but I view "unset" as false-y.
@brettcannon
Specify `[[package.files]] should be sorted by name
c0d46a1
@brettcannon
Clarify that [[package.build-requires]] is locked
0c45e87
@brettcannon
Clean up a sentence
ce18c5c
@brettcannon
Fix a section title
f5b574e
@brettcannon
Tweak abstract
17ce7cd
@brettcannon
Merge branch 'main' into lock-file
6df00c5
@brettcannon
Copy link
Member Author

Thanks to everyone and the nearly 60 (!) review comments. I think I either accepted or answered everyone. If I missed something then let me know and I can fix it in a subsequent PR.

@brettcannon brettcannon merged commit 92634ee into python:main Jul 24, 2024
@brettcannon brettcannon deleted the lock-file branch July 24, 2024 23:13
@JonZeolla JonZeolla mentioned this pull request Jul 26, 2024
Closed
pauleveritt pushed a commit to pauleveritt/peps that referenced this pull request Aug 6, 2024
@brettcannon @hugovk
@AA-Turner @JelleZijlstra @willingc @pauleveritt
PEP 751: A file format to list Python dependencies for installation r…
482f224 
…eproducibility (python#3870)

Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Carol Willing <[email protected]>
jimbaker added a commit to jimbaker/peps that referenced this pull request Aug 7, 2024
@pauleveritt @jimbaker
@lysnikolaou @davidfstr @warsaw @hugovk @erlend-aasland @AA-Turner @Yhg1s @woodruffw @JelleZijlstra @carljm @Rnbsov @Mariatta @brettcannon @willingc @encukou @vstinner @befeleme @hroncok
Last two comments from Carol (#6)
fe012b6 
* Include three comments from @Carreau (drop two bullets, *args in example, explain *args.)

* Lambda-wrapped expressions use annotation scope

* Clarify use of annotation scope

* Mention what happens to named unicodes followed by text

* Use DecodedConcrete in assertion

* Rewrite why annotation scope is needed (#4)

* Rewrite why annotation scope is needed

* Minor copyediting

* PEP 747: Fix rules related to UnionType (T1 | T2). Contrast TypeExpr with TypeAlias. Apply other feedback. (python#3856)

* PEP 694: Fix typo (python#3859)

* PEP 2026: Update following discussion (python#3860)

Co-authored-by: Erlend E. Aasland <[email protected]>

* PEP 101: Remove outdated info and add new info (python#3863)

* PEP 101: Remove outdated info

* PEP 101: Update make command for running tests

* PEP 101: Replace '#python-dev and/or python-committers' with 'Discord and/or Discourse

* PEP 101: Add Hugo as 3.14 RM

* PEP 101: Add to PSRT

* PEP 11: Add Russell as an iOS contact (python#3865)

* Meta: Document the PEPs API (python#3864)

Co-authored-by: Adam Turner <[email protected]>

* PEP 719: Update for today's release of 3.13.0b4 (python#3868)

* PEP 740: Mark as Provisional (python#3848)

Signed-off-by: William Woodruff <[email protected]>

* PEP 749: Add section on metaclasses (python#3847)

Co-authored-by: Carl Meyer <[email protected]>

* PEP 8: Update a Wikipedia link (python#3552)

* PEP 635: Minor typo fix in code sample (python#3871)

Looks like an unclosed f-string.

* PEP 751: A file format to list Python dependencies for installation reproducibility (python#3870)

Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Carol Willing <[email protected]>

* PEP 743: Rewrite to hide (soft-)deprecated API (pythonGH-3869)

Co-authored-by: Victor Stinner <[email protected]>

* PEP 751: Add Discussions-To and Post-History (python#3872)

* PEP 639: Incorporate the latest discussion feedback (python#3866)

* Remove the requirement of license-files defaults

* Cover all rejected subkeysideas in one paragraph

* Change the deprecation policy around classifiers

* Flatten the value of the license-files key, only globs are specified

* Update the Rejected ideas to match the current license-files proposal

---------

Co-authored-by: Miro Hrončok <[email protected]>

* PEP 715: clarify what `[package.tool]` is (python#3873)

* PEP 665: Superseded-By: 751 (python#3875)

* PEP 751: update based on feedback (python#3877)

* PEP 751: update based on feedback

* Fix a section underline

* Include three comments from @Carreau (drop two bullets, *args in example, explain *args.)

* From Carol, move the point about import to the following paragraph.

* Per Carol: Remove paragraph about lifecycles as that is about *a* DSL, not DSLs in general.

---------

Signed-off-by: William Woodruff <[email protected]>
Co-authored-by: pauleveritt <[email protected]>
Co-authored-by: Jim Baker <[email protected]>
Co-authored-by: Lysandros Nikolaou <[email protected]>
Co-authored-by: David Foster <[email protected]>
Co-authored-by: Barry Warsaw <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Erlend E. Aasland <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: T. Wouters <[email protected]>
Co-authored-by: William Woodruff <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Carl Meyer <[email protected]>
Co-authored-by: Lavrentiy Rubtsov <[email protected]>
Co-authored-by: Mariatta <[email protected]>
Co-authored-by: Brett Cannon <[email protected]>
Co-authored-by: Carol Willing <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
Co-authored-by: Karolina Surma <[email protected]>
Co-authored-by: Miro Hrončok <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JelleZijlstra JelleZijlstra JelleZijlstra left review comments

@hugovk hugovk hugovk left review comments

@willingc willingc willingc left review comments

@AA-Turner AA-Turner AA-Turner left review comments

Assignees
No one assigned
Labels
new-pep A new draft PEP submitted for initial review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@brettcannon @JelleZijlstra @hugovk @willingc @AA-Turner