pip install installing from extra-index-url first

[theherk]

I have a package in a private repository named the same name as one in PyPI. I have my index-url set to my private repository and extra-index-url set to PyPI.

My package is version 2.1.0, the one in PyPI 2.2.0.

When I install my package>=2.0.0 it gets 2.2.0 from PyPI. I believe it should install the highest version greater than 2.0.0 in the index-url if it exists there. Pinning the specific version causes it to honor the order of the indices.

The behavior that it installs the highest version in any repo its found in, no matter the order, is a little counter-intuitive.

[Ivoz]

The behavior that it installs the highest version in any repo its found in, no matter the order, is a little counter-intuitive.

I don't think so. Install the latest version amongst the indexes given.

[theherk]

If you have multiple indices, it isn't because you are storing some
versions in one and different versions in the other. It is because you want
to control from which index the installation happens. The highest version
in all indices is totally arbitrary, and doesn't allow any control over the
behavior.

[pfmoore]

If you have multiple indices, it isn't because you are storing some
versions in one and different versions in the other. It is because you want
to control from which index the installation happens. The highest version
in all indices is totally arbitrary, and doesn't allow any control over the
behavior.

That's not the intent of the index arguments. They are to allow extra
indexes for local mirrors, or for repositories of additional packages not
on PyPI.

Using the same package name for 2 different things is not supported.

[piotr-dobrogost]

Compare with issue #928 and issue #3299
I think I remember @dstufft's comment where he said that allowing to set order of preference for different sources of packages would be nice to have.

[xavfernandez]

For the record, I agree with pfmoore

[daisylb]

for repositories of additional packages not on PyPI.

FWIW, this is our usecase at work and Pip's behaviour here is still a bit worrying for us - if someone creates a distribution on PyPI with the same name as one of our internal ones, whether by malice or (more likely) by coincidence, we could end up installing their code instead of ours.

[asobrien]

@adambrenecki Some private index servers (like devpi) get around such an issues by explicitly requiring PyPi packages to be whitelisted (http://doc.devpi.net/latest/userman/devpi_indices.html#modifying-the-mirror-whitelist).

[martinpengellyphillips]

I'm encountering the same situation as @adambrenecki. Ideally, we would not have name clashes, but registering empty packages on PyPI to reserve names for internal projects is not practical at present.

Thanks for tip about devpi - it does seem the best way to go at present.

[dstufft]

I'm going to close this, this is operating as designed and a solution like DevPI is currently the best option. Trying to get more complex then what we have now with our current system is unlikely to be something that we are willing to do.

[theherk]

I understand the resistance to implementing things that a either very difficult or otherwise hurt the operation of a system. In this case, though, I can't see why not to leave this open as a possible improvement down the road. That isn't to say there isn't a cogent reason, but it isn't jumping out at me. It seems simple to add an option to pip.conf to give a preference order for the indices.

I'm not disputing that it is not a bug as it stands; I understand it is working as intended, but the modification would provide benefits and break no contracts with users, I think.

[pfmoore]

Essentially, if anyone feels this is worth having, they can raise a PR. The difference between an issue and a PR is that on a PR we can debate the precise implementation details, whereas on an issue the discussion is essentially theoretical. If no-one is able to create a PR, then there's little or no benefit to having this issue - it's simply contributing to our ever-increasing backlog. As has already been noted in the discussion, none of the core devs are particularly in favour of this suggestion, so leaving it here as a "reminder" for us is pretty pointless.

Also, the only given use case for this is two packages with the same name but unrelated content - that's not something that pip supports anyway (because the relevant standards say names must be unique). If you're concerned about clashes popping up on PyPI, then that's a different problem. You could use a naming convention to limit the risk, disable PyPI searching, propose a reserved namespace that is not allowed on PyPI (local.* or something) or just deal with the issue on a case by case basis should it arise. It's not likely to be a silent failure! But in any case, it's not something we'd solve in this issue.

[mwarkentin]

We currently have a name clash between a private package we have hosted on Gemfury and an open-source package hosted on pypi, however they're only at version 0.1.0 and doesn't seem to be actively developing, so we seem to be ok for now.

This is a key package used across dozens of applications so this behaviour is definitely concerning for us as well. Some way to specify that this package should only be installed from our private repository would be helpful.