resolving TUF target name from distribution download URL

[jku]

A design goal is to minimize the required client configuration. In practice I'm hoping I won't have to store package base directory ('packages/' on files.pythonhosted.org) in the configuration.

The plan is to integrate tuf into pip in a place where we get a Link object which contains among other things the full url of the file to be downloaded and helper properties for parsing it. The issue is how to extract the TUF metadata name from the url?

Example URL:

https://files.pythonhosted.org/packages/ca/ab/5e004afa025a6fb640c6e983d4983e6507421ff01be224da79ab7de7a21f/Django-3.0.8-py3-none-any.whl#sha256=5457fc953ec560c5521b41fad9e6734a4668b7ba205832191bbdff40ec61073c

We want to extract

ca/ab/5e004afa025a6fb640c6e983d4983e6507421ff01be224da79ab7de7a21f/Django-3.0.8-py3-none-any.whl

• With knowledge of base package directory this is easy... Should warehouse include that info in custom metadata? or can we just assume it's always "packages/"?
• Another option is to define that Metadata name is the filename without fragments with enough preceding path components to form the hash: this assumes we know how long the hash is (either warehouse must to tell us or we are not future proof for hash length changes)
• Alternatively define that Metadata name is the filename without fragments with 3 preceding directories -- this is not great for future proofing (but I have no idea if the three directory levels could in practice become too few in the future)

[woodruffw]

• Metadata name is the filename without fragments with enough preceding path components to form the hash: this assumes we know how long the hash is (either warehouse must to tell us or we are not future proof for hash length changes)

This sounds reasonable to me -- it's unlikely that Warehouse will be moving away from BLAKE2b anytime soon, so this could just be a constant that's baked into pip. Then again, it certainly would make migrating a pain, should that ever need to occur. But I expect that such a migration would require a total turnover of the package index links anyways, so perhaps that's not a big deal.

[jku]

Then again, it certainly would make migrating a pain, should that ever need to occur. But I expect that such a migration would require a total turnover of the package index links anyways, so perhaps that's not a big deal.

In the very unlikely event of hash change happening, the worst outcome is that clients that did not upgrade in time would have to use "--disable-package-security" (or whatever it will be called) once to upgrade pip... so maybe this is reasonable

For now, I'll work with the assumption that Metadata name is the filename without fragments with enough preceding path components to form a 256-bit blake2b hash. Thanks for reply.

[jku]

Current implementation is metadata name is filename plus 3 preceding directory names -- but there is a sanity check to ensure the result has the correct length.

I'm closing this but keeping a note to mention it in review