StsCredentialsProvider uses excessive number of threads in multi-tenant setup

[cloudshiftchris]

Describe the bug

When configuring StsCredentialsProvider (or it's subclasses, e.g. StsAssumeRoleCredentialsProvider) for async credential refreshing, an instance of software.amazon.awssdk.utils.cache.NonBlocking is created for each credentials provider. This further creates a ScheduledThreadPoolExecutor with a single thread to handle the async refreshing of credentials.

In a multi-tenant environment where a credentials-provider-per-tenant is used (to provide scoped-down per-tenant IAM policies, e.g https://aws.amazon.com/blogs/apn/isolating-saas-tenants-with-dynamically-generated-iam-policies/) there are a proliferation of ScheduledThreadPoolExecutors (each with a single thread). Threads are expensive resources to create and have laying around at scale.

It isn't necessary to have a thread-per-credentials-provider - a shared ScheduledThreadPoolExecutor with a small pool size would suffice.

Expected Behavior

Scaling the use of StsCredentialsProvider in a multi-tenant environment doesn't consume excessive/unnecessary thread resources.

Current Behavior

A background thread is created for each instance of StsCredentialsProvider for async refresh of credentials. Threads are expensive resources to create and have laying around at scale. Each thread consumes memory (thread stack), and there are hard limits to the number of threads that can be created (these vary based on OS, configuration, and other use of threads inside an app).

Reproduction Steps

n/a. This is a non-functional/scalability design defect, evident in a cursory review of the StsCredentialsProvider code.

Possible Solution

Allow for StsCredentialsProvider builder to take an (optional) ScheduledThreadPoolExecutor (and remove cleanup logic in NonBlocking for externally provided Executors).

This allows consumers, in advanced use cases, to manage the background threads efficiently and avoid resource-starvation scenarios at scale.

Additional Information/Context

No response

AWS Java SDK version used

2.7.214

JDK version used

17

Operating System and version

Mac OS Catalina

[debora-ito]

@cloudshiftchris thank you for the detailed report. Although we don't have a lot of experience on the multi-tenant model, your suggested solution makes sense from the perspective of a product feature.

I see you submitted a PR, we'll take a look.

[debora-ito]

Changed to feature-request since it's an optimization.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.