Add an API to generate registry index data

[carols10cents]

See RFC 2141 for more details/context.

Currently, the knowledge of what goes in the registry index in what format lives in crates.io:

• here,
• which serializes an instance of this struct,
• which comes from here,
• most of which comes from cargo anyway, around here,
• which deserializes an instance of this struct
• which has a corresponding struct in cargo
• that gets created here

In order to support minimal registries, we'd like cargo to be able to generate the JSON information that should go into a registry for the current crate.

Cargo does already understand how to read these entries! We need to teach it how to create them as well :)

So as I understand it, this would involve:

• Derive Serialize for the RegistryPackage struct
• Create a new command, feel free to suggest another name, the first thought in my head is generate-index-metadata, that should:
  • Create an instance of RegistryPackage for the current crate, reusing code that sends relevant information to crates.io when appropriate
  • Serialize that struct to stdout

[carols10cents]

In order to test out #4506, I've created a minimal registry! The part that this proposed command would create is this file's contents, given the current crate.

I don't think we envision people running this command and doing something manually with the output, though that will be possible.

The main goal is to enable any tool to be able to create the same registry entries that crates.io creates right now, without needing to run all of crates.io.

[stephanbuys]

@carols10cents from what I can tell #4506 deals with cargo support for alt-registries, not this issue (ie. generating the actual minimal registry data). I'd like to have a stab at it if this command is still up for grabs.
Does it make sense to abstract some of this work out into a 3rd party crate to be used by crates.io and other registry implementations?

[carols10cents]

@stephanbuys

@carols10cents from what I can tell #4506 deals with cargo support for alt-registries, not this issue (ie. generating the actual minimal registry data).

Yes, #4506 is only tangentially related-- someone on gitter was asking for an example of what a "minimal registry" would look like and what this command needs to generate, and #4506 had just happened to prompt me to do just that :)

Does it make sense to abstract some of this work out into a 3rd party crate to be used by crates.io and other registry implementations?

Cargo can be used as a dependency, and someone on gitter also suggested adding a public API function that returns the text this command would output, so I think that would work. Does that sound ok?

[liranringel]

I started to look into it.
RegistryPackage contains DependencyList which manually implements Deserialize, so I guess it's also necessary to manually implement Serialize.
Good chance to learn some serde.

[liranringel]

What would be the best way to calculate the checksum?
I thought to use cargo::ops::package to get the .crate file.

Edit: I noticed that the hash changes on every cargo package command, perhaps a timestamp is inserted, so this is meaningless to get the checksum.

[withoutboats]

@carols10cents Do you have strong opinions on whether this should live as a cargo subcommand or a separate package we publish to crates.io? My only concern is that cargo could become one of those tools where you type cargo --help and you're drowned in dozens of niche subcommands & you can't find what you're looking for. Do you think its too much to ask people to cargo install this?

[carols10cents]

I guess I don't feel super strongly about whether this is part of cargo or external, but it should be in the rust-lang org and it's seeming like having a library with a public api function as well as a binary would be useful...

[withoutboats]

I definitely agree about both of those points.

[liranringel]

So where should it be?
To make it clear, I interpret the task as getting a .crate file and generating it's index metadata.
If it won't be in cargo, it will probably be in a crate that depends on cargo, or cargo will depend on it (large refactor).