Transfer Manager

[millems]

Review the inherited state of the V1 transfer manager and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

[abrooksv]

Upload and download return a Transfer which has getProgress(). This returns a simple way to get a percent complete but requires you to do busy loops to update things like UIs.

On the other hand, ProgressListener only returns the bytes transferred causing you to have to do the percentage manually and not very easy since the total size is not exposed.

Better feature parity on the ProgressListener would be nice since that plays better with async

[millems]

+1 on parity with progress listener. There should probably just be one system between transfer manager and the rest of the SDK for monitoring progress.

[millems]

Use DirectoryStream for loading files from a directory to avoid having to load all file names into memory. See aws/aws-sdk-java#1271.

[kiiadi]

Unreasonable to expect the thread-pool will be unbounded in order to avoid deadlocks see : aws/aws-sdk-java#939

[spfink]

TransferManager features requests from v1:

aws/aws-sdk-java#117
aws/aws-sdk-java#284
aws/aws-sdk-java#474
aws/aws-sdk-java#645
aws/aws-sdk-java#893
aws/aws-sdk-java#964
aws/aws-sdk-java#988
aws/aws-sdk-java#1215
aws/aws-sdk-java#1207
aws/aws-sdk-java#1103

[millems]

aws/aws-sdk-java#1321

[millems]

Allow using a finite number of threads for background processing. Currently, 1.11.x's TransferManager is reported to require an unbounded thread pool to prevent deadlocks.

[erikedlund]

+1 for aws/aws-sdk-java#1103, a request for the ability to limit bandwidth for S3 uploads/downloads. See also the recently closed issue from the aws-cli repo:
aws/aws-cli#1090

This same feature would be similarly useful in the Java SDK to help avoid fees from ISPs for excessive bandwidth usage, or to prevent a single application from overwhelming a network's capacity.

[zhiqiangZHAO]

+1 for aws/aws-sdk-java#1103, too fast data downloading will saturate the network usage.

[sql4bucks]

+1 for aws/aws-sdk-java#474.

It is very inefficient to write to the file system then upload from the file when I have an object in memory I can serialize directly to a stream. It seems counterintuitive to provide the total content length up front when providing a stream as input - I have to work around it instead of just use it.