pip install from stdin

[kenahoo]

I see that an existing ticket has been rejected: #3880 . I'm requesting that the team reconsider.

The main use case I have for this feature is to build an environment using a set of requirements specified in a URL or coming from a specific git commit:

% curl https://foo.com/requirements.txt | pip install -r -
% git show deadbeef:requirements.txt | pip install -r -

There are workarounds, e.g. to leverage some shell feature for running a subprocess (e.g. pip install -r <(curl https://foo.com/requirements.txt) or similar) or download the file in advance to some temporary location, but these are more esoteric, shell-dependent, or require extra cleanup steps. I believe the cleanest solution for the user is simply to behave like a standard command-line tool and use stdin/stdout in the normal ways.

In the spirit of conforming even better to standard POSIX style, I'd further recommend that the - be unnecessary, e.g. if no file argument is given for -r that input be taken from stdin, just like other input-consuming tools like cat, cut, less, python etc. do:

% curl https://foo.com/requirements.txt | pip install -r
% git show deadbeef:requirements.txt | pip install -r

Thanks for considering.

[pfmoore]

Strong -1 on the -r without arguments form, that seems likely to be very confusing.

I can see the value of -r -, although I doubt I'd use it much myself. There are also likely to be design questions that need to be thrashed out, too. But the biggest blocker is probably that someone needs to actually do the work of writing a PR for this (which includes tests and documentation).

@pypa/pip-committers Is anyone actively against this new feature? I don't want to suggest that someone should go to the effort of writing a PR if it's just going to get rejected.

I've marked the request as "deferred till PR" - I doubt there's much more to discuss until there's actual code to base a discussion on.

[xavfernandez]

Also -1 on the -r without argument.

Concerning -r - I'd be more amenable, but then we'd also have to accept -c - and of course define what should happen for pip install -c - -r -.

[uranusjr]

From personal experience working with other software, a common practice is to accept stdin separated by EOF (^D in readline) by argument order. I have zero idea how this should be implemented though, since optparse does not keep argument ordering AFAIK.

[wchargin]

Also −1 on -r without argument, but definitely +1 on -r -. I just
assumed that this would work and was quite surprised when it didn’t. (My
use case: grep flake8 requirements.txt | pip install -r - in a lint
step.) The existing workarounds are not good; /dev/stdin does not work
on Windows, and <(...) does not work in a POSIX shell.

[duckinator]

I think, if this is added, -r - is the way to go. -r may be more "POSIX style", but it's also an aspect of "POSIX style" programs that I have repeatedly been bamboozled by. 😅

@xavfernandez thank you for pointing out that potential complication. I didn't even know constraints files existed until reading your comment. 🙂

[kenahoo]

I think, if this is added, -r - is the way to go. -r may be more "POSIX style", but it's also an aspect of "POSIX style" programs that I have repeatedly been bamboozled by. 😅

Just to be a POSIX defender: I don't think we'd enjoy a world quite as much where we routinely had to write grep foo file.txt | cut -f2-5 - | sort -r - | less - instead of simply grep foo file.txt | cut -f2-5 | sort -r | less. The cognitive load of | less - instead of | less alone seems like a pretty good validation of the philosophy that "everything works on pipes".

That said, I don't need to debate the philosophy, it would just be nice if we had something in pip that worked easily for this use case.

[pradyunsg]

Added more labels, to clarify state.

@pypa/pip-committers Is anyone actively against this new feature?

Not me.

As far as I can tell:

• we're OK to use - as an indicator of "use stdin", for -r and -c
  • need to define what should happen for pip install -c - -r -
• we DO NOT want to "fall back to stdin" if no arguments are given to -r

[pfmoore]

need to define what should happen for pip install -c - -r -

Personally, I'd just say it's not allowed. It's extremely non-obvious what the best approach is, and it's a pretty obscure corner case. I'd actually be perfectly happy to just call YAGNI on -c - and stick with -r - alone, but there's bound to be someone who wants it "for consistency"...

[sbidoul]

An alternative that works today (example with bash):

pip install $(curl https://foo.com/requirements.txt)