Add pylock #900

sbidoul · 2025-05-06T20:58:13Z

Here is the proposal for pylock.toml support.

immutable keywords-only dataclasses
a from_dict class method to validate and create a Pylock instance from a toml dict (obtained from tomllib.load)
a Pylock.to_dict method to convert to a spec-compliant toml dict (assuming types were respected when populating the dataclass), preserving the recommended field ordering
this uses frozen keyword-only dataclasses, which makes the constructors look weird, but that will be cleaned up without API change when this projects drops support for python 3.9
is_valid_pylock_path to validate pylock file names
toml (de)serialization left out on purpose

Documentation is still TODO.

closes #898

sbidoul · 2025-05-06T21:10:21Z

src/packaging/pylock.py

+    url: str | None  # = None
+    path: str | None  # = None
+    size: int | None  # = None
+    upload_time: datetime | None  # = None


In the spec, the upload_time field is not at the same position for archive and sdist / wheel.

Is that intended or something we want to tweak here and/or in the spec?

That was intended as the idea (and expected value) comes from the index API and I don't think archives are supported there.

But if people have uses for an upload time for archives then I suspect making it a 1.1 change wouldn't be controversial.

Note my remark was only about the ordering of the fields, not about the presence of upload_time in archive.

My idea was to have from_dict independent of the class to cope for future evolution, but if a new version of the standard can't be implemented as a subclass of Pylock, return type of from_dict would change, so it would be a breaking change. If/when a new version of the spec arrive, it will still be time to design the API evolution, depending of the actual changes.

sbidoul · 2025-07-06T16:55:57Z

I consider this PR ready for review. I plan to add documentation after a first round of review, assuming the overall design is agreeable to packaging maintainers, of course.

brettcannon · 2025-07-10T17:41:08Z

FYI I'm hoping to look at this post-EuroPython in hopes that it helps unblock pip so it can add installation support for pylock.toml.

brettcannon

Looks good overall! Some docstring tweaks-- since that will be documentation --and potential test tweaks. You can feel free to ignore the assignment expression suggestions if you want.

brettcannon · 2025-07-25T17:31:19Z

src/packaging/pylock.py

+PYLOCK_FILE_NAME_RE = re.compile(r"^pylock\.([^.]+)\.toml$")
+
+
+def is_valid_pylock_path(path: Path) -> bool:
+    return path.name == "pylock.toml" or bool(PYLOCK_FILE_NAME_RE.match(path.name))


If the regex is not going to be private, we might as well make it work for all names.

Suggested change

PYLOCK_FILE_NAME_RE = re.compile(r"^pylock\.([^.]+)\.toml$")

def is_valid_pylock_path(path: Path) -> bool:

return path.name == "pylock.toml" or bool(PYLOCK_FILE_NAME_RE.match(path.name))

PYLOCK_FILE_NAME_RE = re.compile(r"^pylock\.(?:([^.]+)\.)toml$")

def is_valid_pylock_path(path: Path) -> bool:

return bool(PYLOCK_FILE_NAME_RE.match(path.name))

I wasn't planning to make the regex public, and it's currently not in __all__, so it's kind of private?.

I prefixed the regex and some other things with _.

brettcannon · 2025-07-25T17:32:00Z

src/packaging/pylock.py

+
+
+def _toml_value(key: str, value: Any) -> Any:
+    if isinstance(value, (Version, Marker, SpecifierSet)):


Suggested change

if isinstance(value, (Version, Marker, SpecifierSet)):

if isinstance(value, {Version, Marker, SpecifierSet}):

This does not seem to work, at least in Python 3.13.

src/packaging/pylock.py

tests/test_pylock.py

brettcannon · 2025-07-25T19:06:26Z

tests/test_pylock.py

+    }
+    with pytest.raises(PylockValidationError) as exc_info:
+        Pylock.from_dict(data)
+    assert str(exc_info.value) == "Invalid version: '2.x' in 'lock-version'"


Probably better to just test that "2.x" is somewhere in the message.

Same goes for any of the other tests that's doing an exact exception message comparison instead of just testing the key details are somewhere in the string.

Hmm, I'm not sure about that, since these also test that the message contains the correct context explaining where the error occurs.

tests/test_pylock.py

brettcannon · 2025-07-25T19:11:42Z

tests/test_pylock.py

+    run-on = 2025-03-06T12:28:57.760769
+    """  # noqa: E501
+)
+


Is it worth having a test to verify this example deserializes as expected?

I'd say test_toml_roundtrip does it implicitly? Or do you have something else in mind?

Co-authored-by: Brett Cannon <[email protected]>

Also make _get_*_as function more versatile by accepting a callable for the target type.

These comments were remnants of dataclass experiments.

This is more consitent than having some validations done in the constructor and other in from_dict.

sbidoul · 2025-07-27T12:52:54Z

@brettcannon thanks for the review. I handled the comments and added a documentation page. This is ready for a second round.

sbidoul · 2025-07-28T15:45:01Z

src/packaging/pylock.py

+
+    def validate(self) -> None:
+        """Validate the Pylock instance against the specification."""
+        self.from_dict(self.to_dict())


Note to self: we may want to catch Exception out of to_dict first and re-raise as PylockValidationError. from_dict should only raise PylockValidationError already.

sbidoul mentioned this pull request May 6, 2025

Standard APIs for PEP 751? #898

Open

Add pylock

9230293

sbidoul force-pushed the pylock branch from 592c430 to 9230293 Compare May 6, 2025 21:07

sbidoul commented May 6, 2025

View reviewed changes

sbidoul mentioned this pull request May 7, 2025

Add pylock parser and validator pypa/pip#13369

Draft

2 tasks

sbidoul marked this pull request as draft May 7, 2025 08:23

sbidoul marked this pull request as ready for review May 7, 2025 08:23

pylock: clarify error messages

2343f8a

sbidoul force-pushed the pylock branch from 048d455 to 2343f8a Compare May 7, 2025 09:31

pylock: simplify

5ab8ae9

sbidoul force-pushed the pylock branch from f3fc4ad to 5ab8ae9 Compare May 7, 2025 09:44

This was referenced May 9, 2025

Replace vendored packaging with PyPI package once a version with pylock support is released trailofbits/pylock-attestations#3

Open

Support for PEP 751 trailofbits/pypi-attestations#112

Open

brettcannon self-requested a review July 25, 2025 17:22

brettcannon requested changes Jul 25, 2025

View reviewed changes

sbidoul force-pushed the pylock branch 5 times, most recently from 44f8995 to 3bbb50d Compare July 27, 2025 09:16

pylock: Apply suggestions from code review

893bc7b

Co-authored-by: Brett Cannon <[email protected]>

sbidoul force-pushed the pylock branch from 85e004c to a36f9cb Compare July 27, 2025 10:46

sbidoul added 5 commits July 27, 2025 12:49

pylock: type name as NormalizedName

3a4c2ed

Also make _get_*_as function more versatile by accepting a callable for the target type.

pylock: type extras as NormalizedName, and validate

6102a1a

pylock: move PEP751 example to a standalone file

79f4432

pylock: make more module level stuff "private"

becaf04

pylock: re-enable default None values

3686eb0

These comments were remnants of dataclass experiments.

sbidoul force-pushed the pylock branch from a36f9cb to 3686eb0 Compare July 27, 2025 10:52

sbidoul added 3 commits July 27, 2025 13:55

pylock: add validate method

6abf8cf

pylock: validate everything in from_dict

4f310b2

This is more consitent than having some validations done in the constructor and other in from_dict.

pylock: add documentation

bc61305

sbidoul force-pushed the pylock branch from a59d102 to bc61305 Compare July 27, 2025 12:48

sbidoul commented Jul 28, 2025

View reviewed changes



		def _toml_value(key: str, value: Any) -> Any:
		if isinstance(value, (Version, Marker, SpecifierSet)):

	if isinstance(value, (Version, Marker, SpecifierSet)):
	if isinstance(value, {Version, Marker, SpecifierSet}):

Add pylock #900

Are you sure you want to change the base?

Add pylock #900

Uh oh!

Conversation

sbidoul commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbidoul commented Jul 6, 2025

Uh oh!

brettcannon commented Jul 10, 2025

Uh oh!

brettcannon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbidoul commented Jul 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbidoul commented May 6, 2025 •

edited

Loading