Skip to content

Conversation

millems
Copy link
Contributor

@millems millems commented Dec 3, 2020

Most changes are from a rename of MockHttpClient to MockSyncHttpClient so that a common parent interface can be added for generic programming purposes.

The new amz-sdk-request header does not include the TTL, because it's not at all easy to calculate with pluggable HTTP clients and the specification lists it as optional. The TTL can be added when service teams ask for it.

return request.toBuilder()
.putHeader(SDK_RETRY_INFO_HEADER, headerValue)
.putHeader(SDK_RETRY_INFO_HEADER, "attempt=" + attemptNumber + "; max=" + retryPolicy.numRetries())
Copy link
Contributor

@zoewangg zoewangg Dec 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the retry capacity is very valuable for troubleshooting; otherwise there's no way to tell if the retry gets throttled. I've seen an issue before where all requests failed of IOException, and I initially thought all retry attempts failed, but this header made me realize that the request did not get retried because there was no retry capacity.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess now that we have metrics, we can see it there. It's just not as obvious

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could add a debug log whenever we do not retry because of throttling?

@millems millems force-pushed the millem/retry-headers branch from 5179a53 to 5d9c56a Compare December 4, 2020 18:44
Most changes are from a rename of MockHttpClient to MockSyncHttpClient so that a common parent interface can be added for generic programming purposes.

The new amz-sdk-request header does not include the TTL, because it's not at all easy to calculate with pluggable HTTP clients and the specification lists it as optional. The TTL can be added when service teams ask for it.
@millems millems force-pushed the millem/retry-headers branch from 5d9c56a to 1cffce8 Compare December 4, 2020 18:48
@@ -122,9 +125,17 @@ public boolean shouldRetry(RetryPolicyContext context) {
context.executionAttributes().putAttribute(LAST_ACQUIRED_CAPACITY, c);
context.executionAttributes().putAttribute(RETRY_COUNT_OF_LAST_CAPACITY_ACQUISITION,
context.retriesAttempted());
log.trace(() -> "Successfully acquired token bucket capacity to retry this request. "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seemed like a bit much for debug, since we're already logging when there wasn't enough capacity. I was thinking if we really needed to debug this, we can ask them to enable trace for this one class.

/**
* A set of tests that verify the behavior of retry-related headers (amz-sdk-invocation-id and amz-sdk-request).
*/
public abstract class RetryHeaderTestSuite<T extends MockHttpClient> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@millems millems merged commit 0e016da into master Dec 7, 2020
@millems millems deleted the millem/retry-headers branch December 7, 2020 18:56
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 7, 2020

aws-sdk-java-automation added a commit that referenced this pull request Oct 5, 2022
…2778682e7

Pull request: release <- staging/d25ebd20-7ff7-4143-b9ba-3222778682e7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants