Skip to content

Adding custom backoff function for API's that _can_ be excessive #354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 1, 2025

Conversation

davidcollom
Copy link
Collaborator

During testing it became apparent that version-checker would wait and honour the "Retry-After" Header and wait almost in definatly for the next request if a response was given such as below:


INFO[0036] [DEBUG] GET https://registry.hub.docker.com/v2/repositories/xxxxxx/yyyyy/tags?page_size=100  client=docker component=client
INFO[0036] [DEBUG] GET https://registry.hub.docker.com/v2/repositories/xxxxxx/yyyyy/tags?page_size=100 (status: 429): retrying in 484289h20m8s (10 left)  client=docker component=clientz

Lets not wait for 484289h20m8s !!!

func HTTPBackOff(min, max time.Duration, attemptNum int, resp *http.Response) time.Duration {
sleep := retryablehttp.DefaultBackoff(min, max, attemptNum, resp)
if sleep.Abs() <= max {
return sleep.Abs()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor code legibility - this is backwards to how I'd normally read code. I'd normally expect the conditional to handle the exception case, here we're returning what I'd expect would be the normal mode of operation.

retryclient.RetryWaitMin = 1 * time.Second
// This custom backoff will fail requests that have a max wait of the RetryWaitMax
retryclient.Backoff = util.HTTPBackOff
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth adding a note here that this could be removed if hashicorp/go-retryablehttp#247 is addressed.

@davidcollom davidcollom merged commit b0ca912 into main May 1, 2025
5 checks passed
@davidcollom davidcollom deleted the custom-backoff branch May 1, 2025 08:09
@github-actions github-actions bot mentioned this pull request Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants