Interest in supporting PEP 665?

[brettcannon]

What's the problem this feature will solve?

Standardizing on a file format that fixes some shortcomings of requirements files.

Describe the solution you'd like

Two possibilities.

One, pip freeze emits PEP 665 files.

Two, -r (or some new option) can read in and install from a PEP 665 file.

Alternative Solutions

Status quo.

Additional context

https://discuss.python.org/t/pep-665-take-2-a-file-format-to-list-python-dependencies-for-reproducibility-of-an-application/11736

I'm basically looking for tool support to quote in the PEP at this point to help drive its support/acceptance.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

I feel pip freeze is a wrong approach, similar to the request for pip freeze --generate-hashes (which we get quasi-often). PEP 665 requires artifact hashes by default, but pip freeze operates on an environment, not artifacts, so those hashes are not available. A better approach would be a new command pip resolve (or say pip install --dry-run but IIRC pip resolve is the more popuilar proposal?) that emits this file from a set of requirements (command line arguments and/or -r).

+1 to making -r accept the PEP 665 format. Also -c. This would also be a good time to add something like --remove-unused to install so it's possible to "sync" the lock file to an environment, instead of simply applying the lock file on top of existing packages (which is the semantic of pip install -r). Alternatively we can also have pip sync just for this.

[pradyunsg]

To answer the interest question: Yes, to the extent that multiple maintainers are co-authors. ;)

[pradyunsg]

For the UI design discussion:

As noted, pip freeze doesn't have the information necessary to generate the lockfile. +1 to having this in pip resolve -- it likely makes sense as the default output format as well.

-1 to allowing this in -r. I'd much rather we create a clearly separate option to consume the lockfile. pip install --lock-file whatever.pylock.toml is clearer, will avoid having conditionals and -l is available. There's no reason to overload an existing option, to have multiple behaviours from my PoV.

I also think it would make sense for pip sync-style behaviour to be the mode of operation for consuming from a lockfile -- I can't think of any good reason to "add" to an environment using a lockfile... so I'd prefer to defer adding that functionality until we have a user express interest in doing things like that (plus, we'll know more about the exact usecase, which is also good).

[uranusjr]

I'd prefer to defer adding that functionality until we have a user express interest in doing things like that (plus, we'll know more about the exact usecase, which is also good).

Arguably we already do? #5823 #9118 #9166 #9799

[pradyunsg]

Hmm... I think there's a misunderstanding. Allow me to try again. What I'm saying we should do is:

• Leave pip install -r [path] as is. It modifies the current environment to ensure that these requirements are available and satisfied; installing and upgrading as necessary.
• Add pip install --lock-file [path]. It'll modify the current environment to match the environment described in the lockfile; installing, upgrading and uninstalling as necessary. This is the pip sync-style behaviour that the issues you've linked to are asking for.

I don't think we should have a mode where you can disable the uninstall semantics when using the lockfile; not unless someone explicitly asks for it. In other words, I think we shouldn't add is "install things from this lockfile to the environment, in addition to whatever is already installed" (think of howpip install -r requirements.txt works right now).

[sbidoul]

+1 for supporting this in pip.

What follows is not much more than instinctive reactions at this stage, as I have not digested the draft PEP yet.

For installation, a -l option to pip install looks good to me. I don't think it should uninstall anything though, as that would be counter-intuitive for an install command.

I'm not sure pip should venture in the environment management space just yet. Although if we want to, pip sync resonates well with me.

Regarding generation of the lock file I always thought that both the pip freeze and pip-compile approaches are valuable. Freeze does require recording information about the original artifact (its hash at least) but that sounds doable although it probably requires a new PEP.

Also regarding the generation of the lock file, I have a question: are there thoughts or examples of how the UI/UX would look like for generating a multi-platform lock file (with pip or in general) ?

[pfmoore]

My instinct is that "installing a lock file" has to mean "making the environment look like what the file specifies". Otherwise there are too many questions around what happens in the case of conflicts, etc. But I agree that it may be confusing if pip install starts doing uninstalls as well.

I think if we're supporting lockfiles, we need a new subcommand (+1 for pip sync) and we need to accept that we're getting into "environment management". There are too many options to pip install that would need thinking through and could cause issues if we have a lockfile, so I think starting with a clean slate would be better.

One possible issue though - in general, I'd expect a lockfile would definitely not specify that pip was to be installed (nothing depends on pip, after all). But we'd have to have pip in the environment to do the install, so "install exactly what is in the lockfile" won't be 100% accurate anyway, we'll need to make an exception for pip. Not a big deal, but let's be careful to be explicit here...

[uranusjr]

Hmm... I think there's a misunderstanding.

Yeah, I think I read your response wrong.

The one thing I don't like about having pip install -l do the sync semantic is it makes pip install -l <file> and pip install -r <file> do different things, which can be confusing for newcomers. It also brings weird combinations like pip install -l <file> [more packages] which obviously does not make sense and we need to manually reject. It definitely makes more sense to me to have a new command doing the new semantic. pip install -l should either match -r, or not exist at all.

[pradyunsg]

Yea, a pip sync commands works too for me -- as long as we're not mimicking the -r behaviours, I'm happy. :)

[pradyunsg]

Also regarding the generation of the lock file, I have a question: are there thoughts or examples of how the UI/UX would look like for generating a multi-platform lock file (with pip or in general) ?

Right now, the workflow that I've got bouncing around in my head is:

$ pip resolve -r requirements.txt mousebender packaging -o something.pylock.toml
$ pip sync something.pylock.toml

We'd effectively be providing the workflow that pip-tools provides; albeit with a standard-backed format (instead of requirements.txt) as the intermediary.