Retry certain failed HTTP requests?

[sjakobi]

As discussed in #1794, it might be useful to retry HTTP requests after certain failures. That might also help with the robustness of the testsuite: #1808 (comment)

[Gabriella439]

Note that for #1794 specifically it appears that retrying would not have helped (it was a persistent DNS resolution problem), but there may still be other cases where retrying is justified. We just need to catalog which http-client exceptions warrant a retry

[sjakobi]

Another instance of a flaky tests due to a network issue: #1825 (comment)

[Gabriella439]

I think it might be fine to make the number of retries configurable (either via the API, the command line, or both), but my intuition is that the default number of retries should still be zero. We would ideally like to fail as quickly as possible when something goes wrong.