Static HTTP client to prevent new connection pools each request

[tobywf]

Issue #, if available:

Description of changes: Static HTTP client to prevent new connection pools each request.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[rjlohan]

Looks good. I still question 16 retries though. Think it might blow past available runtime if we actually retried that many times with exponential backoff.

[rjlohan]

Given the available runtime, is 16 retries too many?

[tobywf]

I just ported it as is, but happy to tweak this if we have better values (the defaults?).

[tyangjie]

+1, default is 3 times I think (except 10 for DDB client). 16 seems too much.

[tyangjie]

Also, since all aws clients are sharing the same HTTP client, do we want to increase the Maximum Connections?

[tobywf]

I think 50 is fine, right? Given that the Lambda in each VM only really serves one request at a time (even if we create a few more connections for every request).

[tobywf]

I'll be exercising handlers based on this all weekend, but so far it's looking promising. The heap size isn't growing, GC is happening, and it looks fine. I'm somewhat confident that this won't accidentally share credentials (the HTTP client doesn't use AWS credentials, so having a long-lived HTTP client should be safer than having long-lived clients).

[tobywf]

Looking good. Will merge on Monday once it's ran all weekend without issues.

[johnttompkins]

may want to also bump the java lib?

[tobywf]

I was going off #213 , which is also linked in the runbook. Is that PR incomplete/what else needs to be done?

[johnttompkins]

I was going off #213 , which is also linked in the runbook. Is that PR incomplete/what else needs to be done?

Since there are changes to the java library we'd need something like [this] (aws-cloudformation/cloudformation-resource-schema#63). The previous PR only had changes to the python lib & the wrapper java library stayed the same, so there was no reason to bump.