Publishing workspaces with large numbers of crates hits the rate limit

[behnam]

When publishing UNIC 0.9.0, there's a sequence of cargo update and cargo publish for all the components (28 of them).

Near the end of publish, at cargo publish --verbose --manifest-path unic/emoji/char/Cargo.toml, I got the following error:

   Executing cargo publish --verbose --manifest-path unic/emoji/char/Cargo.toml

    Updating crates.io index
   Packaging unic-emoji-char v0.9.0 (/Users/behnam/code/open-i18n/rust-unic/unic/emoji/char)
   Archiving Cargo.toml
   Archiving src/emoji.rs
   Archiving src/emoji_component.rs
   Archiving src/emoji_modifier.rs
   Archiving src/emoji_modifier_base.rs
   Archiving src/emoji_presentation.rs
   Archiving src/emoji_version.rs
   Archiving src/lib.rs
   Archiving src/pkg_info.rs
   Archiving tables/emoji.rsv
   Archiving tables/emoji_component.rsv
   Archiving tables/emoji_modifier.rsv
   Archiving tables/emoji_modifier_base.rsv
   Archiving tables/emoji_presentation.rsv
   Archiving tables/emoji_version.rsv
   Archiving tests/emoji_version_tests.rs
   Archiving .cargo_vcs_info.json
   Verifying unic-emoji-char v0.9.0 (/Users/behnam/code/open-i18n/rust-unic/unic/emoji/char)
    Updating crates.io index
   Compiling unic-char-range v0.9.0
   Compiling unic-common v0.9.0
     Running `rustc --edition=2018 --crate-name unic_char_range /Users/behnam/.cargo/registry/src/git.colasdn.top-1ecc6299db9ec823/unic-char-range-0.9.0/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' -C metadata=580cacac1f2fc887 -C extra-filename=-580cacac1f2fc887 --out-dir /Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps -L dependency=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps --cap-lints allow`
     Running `rustc --edition=2018 --crate-name unic_common /Users/behnam/.cargo/registry/src/git.colasdn.top-1ecc6299db9ec823/unic-common-0.9.0/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' -C metadata=4b06076d4a09200d -C extra-filename=-4b06076d4a09200d --out-dir /Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps -L dependency=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps --cap-lints allow`
   Compiling unic-ucd-version v0.9.0
     Running `rustc --edition=2018 --crate-name unic_ucd_version /Users/behnam/.cargo/registry/src/git.colasdn.top-1ecc6299db9ec823/unic-ucd-version-0.9.0/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=c925fe73a5f2ecc8 -C extra-filename=-c925fe73a5f2ecc8 --out-dir /Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps -L dependency=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps --extern unic_common=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps/libunic_common-4b06076d4a09200d.rlib --cap-lints allow`
   Compiling unic-char-property v0.9.0
     Running `rustc --edition=2018 --crate-name unic_char_property /Users/behnam/.cargo/registry/src/git.colasdn.top-1ecc6299db9ec823/unic-char-property-0.9.0/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=916399f60dbb4e87 -C extra-filename=-916399f60dbb4e87 --out-dir /Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps -L dependency=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps --extern unic_char_range=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps/libunic_char_range-580cacac1f2fc887.rlib --cap-lints allow`
   Compiling unic-emoji-char v0.9.0 (/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0)
     Running `rustc --edition=2018 --crate-name unic_emoji_char src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=20bfae06c68c7d3c -C extra-filename=-20bfae06c68c7d3c --out-dir /Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps -C incremental=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/incremental -L dependency=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps --extern unic_char_property=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps/libunic_char_property-916399f60dbb4e87.rlib --extern unic_char_range=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps/libunic_char_range-580cacac1f2fc887.rlib --extern unic_ucd_version=/Users/behnam/code/open-i18n/rust-unic/target/package/unic-emoji-char-0.9.0/target/debug/deps/libunic_ucd_version-c925fe73a5f2ecc8.rlib`
    Finished dev [unoptimized + debuginfo] target(s) in 1.71s
   Uploading unic-emoji-char v0.9.0 (/Users/behnam/code/open-i18n/rust-unic/unic/emoji/char)
error: failed to get a 200 OK response, got 429
headers:
	HTTP/1.1 100 Continue



	HTTP/1.1 429

	Connection: keep-alive

	Server: nginx

	Date: Sun, 03 Mar 2019 05:13:51 GMT

	Content-Length: 0

	Strict-Transport-Security: max-age=31536000

	Via: 1.1 vegur



body:

The client, cargo, seems to not understand the response. Not sure if the server is at fault (with a bad response) or the client (with not handling the response). Anyways, reporting to investigate.

[carols10cents]

I'm really sorry about that! We recently added some rate limiting on the publish endpoint, and 429 is too many requests.

We don't want to prevent publishing multiple crates in a workspace, but we were unsure of what folks doing legitimate publishing would need the limit to be. Now we have a data point!

There are many solutions; we've discussed making the rate limit apply to new crates only rather than new versions, we could slow down requests rather than rejecting them, and the most straightforward, we could increase the rate limit. I'm going to increase the limit to 30 so that you can publish, and I'll put this on our agenda for our next meeting to discuss a better long term solution.

[carols10cents]

Oh also Cargo not displaying a nice error with a clear explanation is a bug that I'll make sure to file in cargo's repo.

[behnam]

Thanks, @carols10cents, for the prompt fix!

Yeah, making sure Cargo shows a nice message would be great. Besides that, I think we're good to close it here.

Side note: we're aware of the fact that re-publishing all UNIC components at once is not ideal. Trying to find a better solution to be able to guarantee certain code and data compatibility without mass-publishing. That said, I think there are still legitimate cases for us to mass-publish, and this fix unblocks those future cases, as well. Thanks! :)

[carols10cents]

@behnam Please try publishing again; the rate limit should now be 30 publishes per minute.

I am going to leave this open because I bet you're not the only project out there, so we do need to figure out some sort of change here.

I also filed a bug with cargo for improving the error display and added this issue to the meeting agenda for next week.

[behnam]

Thanks, Carol! All sounds good.

About UNIC, I was able to publish the few last components with running the commands manually after a while. So, nothing block us at the moment.