SDK ignores "Content-Length" header when sending the body of S3 PUT requests

[yzarubin]

If I create a PUT request, where I incorrectly specify a Content-Length header value LESS than the size of the payload, SDK will actually send the entire payload.

    public static void main(final String[] args) throws InterruptedException {
        final PutObjectRequest r = PutObjectRequest.builder()
                .bucket(BUCKET_NAME)
                .key("test-key")
                .contentLength(1L)
                .build();

        s3.putObject(r, AsyncRequestProvider.fromString("abcd")).join();
    }

The SDK sends the header:

PUT https://<omitted>.s3-us-west-2.amazonaws.com/test-key HTTP/1.1
amz-sdk-invocation-id: <omitted>.
amz-sdk-retry: 0/0/500
Authorization: <omitted>
Content-Length: 1
Expect: 100-continue
Host: <omitted>.s3-us-west-2.amazonaws.com
User-Agent: aws-sdk-java/2.0.0-preview-10-SNAPSHOT Mac_OS_X/10.12.6 Java_HotSpot_TM__64-Bit_Server_VM/25.161-b12 Java/1.8.0_161
x-amz-content-sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20180412T224518Z)

Followed by the body:

         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 62 63 64                                     |abcd            |
+--------+-------------------------------------------------+----------------+

Which results in a 200 Success:

HTTP/1.1 200 OK
x-amz-id-2: <omitted>
x-amz-request-id: <omitted>
Date: Thu, 12 Apr 2018 22:45:20 GMT
ETag: <omitted>
Content-Length: 0
Server: AmazonS3

If I GET the object, it will correctly be of size 1, containing just the character "a". The problem is that if the connection stays open, S3 will keep the remaining "bcd" characters in its buffer. So if I issue a second PUT request (or any request, really), and my previous connection is reused (which it should be), then the "bcd" will be the prefix of whatever headers are sent, which of course is invalid and results in a 400 Bad Request.

Full code:

package bwtest;

import software.amazon.awssdk.core.async.AsyncRequestProvider;
import software.amazon.awssdk.core.auth.AwsCredentials;
import software.amazon.awssdk.core.auth.StaticCredentialsProvider;
import software.amazon.awssdk.core.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;


public class AsyncClientBugMain {

    private static final String BUCKET_NAME = "";
    private static final String ACCESS = "";
    private static final String SECRET = "";
    private static final String REGION = "";
    private static final String TEST_KEY = "test-key";
    private static final String TEST_KEY_2 = "test-key-2";
    private static final S3AsyncClient s3 = S3AsyncClient.builder()
            .credentialsProvider(StaticCredentialsProvider.create(AwsCredentials.create(ACCESS, SECRET)))
            .region(Region.of(REGION))
            .build();

    public static void main(final String[] args) throws InterruptedException {
        final PutObjectRequest r = PutObjectRequest.builder()
                .bucket(BUCKET_NAME)
                .key(TEST_KEY)
                .contentLength(1L)
                .build();

        s3.putObject(r, AsyncRequestProvider.fromString("abcd")).join();

        try {
            final PutObjectRequest r2 = PutObjectRequest.builder()
                    .bucket(BUCKET_NAME)
                    .key(TEST_KEY_2)
                    .build();

            // This will respond with a 400 Bad Request
            s3.putObject(r, AsyncRequestProvider.fromString("efg")).join();
        } catch (final RuntimeException e) {
            e.printStackTrace();
        }
    }
}

• AWS Java SDK version used: 2.0.0-preview-9
• JDK version used: jdk1.8.0_161.jdk
• Operating System and version: Mac Sierra 10.12.6, but also reproduced the problem on the newest EC2 Linux AMI's

[calvernaz]

In your example, you're not using the request "r2" - could that be the problem?

[dagnir]

The fix will be included in our next release (2.0.0-preview-11). Thanks!