TransferManager::DoSinglePartDownload() should flush the stream before marking the transfer completed

[igorcanadi]

TransferManager::DoSinglePartDownload() marks the transfer as completed here: https://github.com/rockset/rocksdb-cloud/blob/8c0ff476d77c0df0af92b1466269404bdfb8e89e/cloud/aws/aws_s3.cc#L196

This happens before the underlying response stream is flushed. The response stream's ownership is tied to local variable getObjectOutcome and the stream is flushed only in getObjectOutcome's destructor when it out of scope (here:

aws-sdk-cpp/aws-cpp-sdk-core/source/utils/stream/ResponseStream.cpp

Line 61 in 5543842

m_underlyingStream->flush();

).

This means that a code that uses std::fstream to download S3 object to a local file and does:

handle->WaitUntilFinished();
// do something with the file

is incorrect, because the file might not be flushed when WaitUntilFinished() returns.

This is similar to this 6 year old bug: #58

[KaibaLopez]

Hi @igorcanadi ,
Yea thanks for bringing attention to this,and you're right, this is an ongoing issue with the SDK, I'll try to make it a priority for the team.

[igorcanadi]

Hi @KaibaLopez any news on this issue? Thanks!

[SergeyRyabinin]

Hi,
I think it should be fixed now by #2351
At least a flush() is called before a task completion notification.
Closing the issue, please submit a new one or re-open should you see more issues.

Best regards,
Sergey

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.