PutObject with ChecksumAlgorithm.SHA256 over unencrypted HTTP results in 400 error

[skand888]

Describe the bug

When we specify ChecksumAlgorithm.SHA256, it's expected that the SDK will calculate the checksum and include it to the request.

However, the following code results in software.amazon.awssdk.services.s3.model.S3Exception: The provided 'x-amz-content-sha256' header does not match what was computed. (Service: S3, Status Code: 400, ...)

        var req = PutObjectRequest.builder()
                .bucket(bucketName)
                .key(objectKey)
                .checksumAlgorithm(ChecksumAlgorithm.SHA256)
                .build();

        var resp = assertDoesNotThrow(
                () -> s3.putObject(req, RequestBody.fromFile(new File("test.txt"))));

It works as expected and completes sucsefully over HTTPS.

The entire request captured with Wireshark:

PUT /objectKey_putObjectWithSha256Algorithm HTTP/1.1
Host: fl-at-jv2-20240812-153626-664046833.s3.ap-southeast-1.amazonaws.com
amz-sdk-invocation-id: 57138d39-4375-a260-8e6b-99e10c058397
amz-sdk-request: attempt=1; max=4
Authorization: [removed]
Content-Type: text/plain
Expect: 100-continue
User-Agent: aws-sdk-java/2.26.29 Linux/5.15.153.1-microsoft-standard-WSL2 OpenJDK_64-Bit_Server_VM/21.0.4+7-LTS Java/21.0.4 vendor/Eclipse_Adoptium io/sync http/Apache cfg/retry-mode/legacy cfg/auth-source#stat ft/s3-transfer
x-amz-checksum-sha256: SMxWlcQBWBPkquqaEFUHaNZMNHb59jFjJ5p1DQueB7c=
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240812T153630Z
x-amz-sdk-checksum-algorithm: SHA256
Content-Length: 28
Connection: Keep-Alive

Actual file SHA256 is different from either SHA256 value:

$ sha256sum test.txt
8422b06904c61e5f05a2cd18add44479c6f4d8be2916adfbeb32eb69ab450b38  test.txt

$ sha256sum test.txt | awk '{print $1}' | xxd -r -p | base64
hCKwaQTGHl8Fos0YrdREecb02L4pFq376zLraatFCzg=

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is SHA256 of an empty zero-size file, SMxWlcQBWBPkquqaEFUHaNZMNHb59jFjJ5p1DQueB7c= is checksum of something else.

Expected Behavior

The code successfully uploads object via both HTTP and HTTPS.

Current Behavior

HTTPS: success
HTTP: software.amazon.awssdk.services.s3.model.S3Exception: The provided 'x-amz-content-sha256' header does not match what was computed. (Service: S3, Status Code: 400, ...)

Other checksum algorithms work fine.

Reproduction Steps

Over HTTP endpoint URL:

        var req = PutObjectRequest.builder()
                .bucket(bucketName)
                .key(objectKey)
                .checksumAlgorithm(ChecksumAlgorithm.SHA256)
                .build();

        var resp = s3.putObject(req, RequestBody.fromFile(new File("test.txt")));

Possible Solution

Properly calculate value for x-amz-checksum-sha256 even when x-amz-content-sha256 is used in signing process.

Additional Information/Context

No response

AWS Java SDK version used

2.26.29

JDK version used

21

Operating System and version

Linux

[bhoradc]

Hello @skand888,

Thank you for reporting the issue. The above S3 Exception - The SHA256 you specified did not match the calculated checksum. is reproducible over non-secure (http) S3 endpoint in Java SDK version 2.21.16 and above.

S3Client s3Client = S3Client.builder()
         .endpointOverride(URI.create("http://s3.us-east-1.amazonaws.com"))
         .build();

var req = PutObjectRequest.builder()
         .bucket("bucketname")
         .key("objectname")
         .checksumAlgorithm(ChecksumAlgorithm.SHA256)
         .build();

Path path = Paths.get("/Users/**/test.txt");
var resp = s3Client.putObject(req, path);

We will investigate further on this.

Regards,
Chaitanya