Closed
Description
Describe the bug
Hello,
I'm trying a simple S3 PUT of an object of size 1MB. Through some reading of the code, I found that the sdk reads the request body at least thrice:
- md5 checksum
- sign the payload
- actually sending the payload
So I decided to:
- turn off signing
- choose CRC32C as the checksum algorithm, which as per this thread is streaming
For small objects (1KB), it works fine. For large object, of size 1MB, I see the following error:
Unable to parse ExceptionName: InvalidChunkSizeError Message: Only the last chunk is allowed to have a size less than 8192 bytes
Expected Behavior
The request should succeed even with custom configurations set that override the default checksum algorithm as well as payload signing policy.
Current Behavior
The PUT request fails with HTTP error code 403 stating InvalidChunkSizeError.
Reproduction Steps
TEST_F(S3Test, crc){
Aws::SDKOptions options;
Aws::InitAPI(options);
Aws::S3::S3ClientConfiguration config;
config.payloadSigningPolicy =
Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never;
auto client = std::make_unique<Aws::S3::S3Client>(config);
Aws::S3::Model::PutObjectRequest request;
request.SetBucket("s3-gtest");
request.SetKey("crctest");
// Use a checksum that is streaming and so doesn't require reading the
// request body twice.
request.SetChecksumAlgorithm(Aws::S3::Model::ChecksumAlgorithm::CRC32C);
std::string s(1024*1024, 'a');
std::stringstream ss;
ss << s;
auto stream = Aws::MakeShared<Aws::IOStream>(nullptr, ss.rdbuf());
request.SetBody(stream);
auto outcome = client->PutObject(request);
if(!outcome.IsSuccess()){
std::cout << "outcome err = " << outcome.GetError().GetMessage() << std::endl;
}
client.reset();
Aws::ShutdownAPI(options);
}
stdout:
outcome err = Unable to parse ExceptionName: InvalidChunkSizeError Message: Only the last chunk is allowed to have a size less than 8192 bytes
Possible Solution
No response
Additional Information/Context
No response
AWS CPP SDK version used
1.11.411
Compiler and Version used
Apple clang version 15.0.0 (clang-1500.3.9.4)
Operating System and version
MacOS 14.4.1