Skip to content

FileStream should flush on network operation complete #58

@cjhanks

Description

@cjhanks

In gist I have code like this..

std::string filePath = "some/local/path";
std::shared_ptr<Aws::Http::HttpRequest>  request;
request->SetResponseStreamFactory(
        [&filePath, &offset]() -> Aws::IOStream* {
            constexpr auto OpenFlag = std::ios_base::out
                                    | std::ios_base::binary;
            Aws::IOStream* ofs =
                Aws::New<Aws::FStream>(ALLOCATION_TAG, filePath, OpenFlag);
            CHECK(ofs->good());
            CHECK(ofs->seekp(offset, ofs->beg));
            return ofs;
        });

std::shared_ptr<Aws::Http::HttpResponse> response(std::move(m_client->MakeRequest(*request)));

std::ifstream ifs(filePath); // ! DATA NOT YET FLUSHED TO FILE HERE

However if I artificially scope everything as follows...

std::string filePath = "some/local/path";
{ // BEGIN BOGUS SCOPE
std::shared_ptr<Aws::Http::HttpRequest>  request;
request->SetResponseStreamFactory(
        [&filePath, &offset]() -> Aws::IOStream* {
            constexpr auto OpenFlag = std::ios_base::out
                                    | std::ios_base::binary;
            Aws::IOStream* ofs =
                Aws::New<Aws::FStream>(ALLOCATION_TAG, filePath, OpenFlag);
            CHECK(ofs->good());
            CHECK(ofs->seekp(offset, ofs->beg));
            return ofs;
        });

std::shared_ptr<Aws::Http::HttpResponse> response(std::move(m_client->MakeRequest(*request)));
} // END BOGUS SCOPE 

std::ifstream ifs(filePath); // DATA IS NOW FLUSHED

This indicates to me a few things:

  • You're not flushing the file stream.
  • You're lazily deleting the Aws::IOstream object

This behavior is surprising and cost me ~45 minute of debug time.

This may /seem obvious/ in this contrived example. But the entire AWS SDK is designed using shared pointers practically everywhere, and as such... it can be come less clear in complicated code what the precise lifetime of objects will be.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions