InputStreamSubscriber changes read() behaviour when passed len=0. Breaks InputStream.readAllBytes

[mgrundie-r7]

Describe the bug

When calling ResponseInputStream.readAllBytes() only 8192 Bytes will be read

I am trying to readAllBytes of a ResponseInputStream to a byte array. The flow of execution for this is shown below:

ResponseInputStream.readAllBytes()
-> InputStream.readAllBytes()
->InputStream.readNBytes(2147483647)
-> FilterInputStream.read(byte[8192] ,0, 8192)
-> InputStreamSubscriber.read(byte[8192], 0, 8192)

When len is set to 0, read() should return 0

InputStream.readNBytes builds up a List of byte arrays (each 8192 bytes) which are chunks read from the InputStream.

To create each new chunk (buf) array, readNBytes expects the result of read(byte[8192] 0, 0) to return 0 as described in InputStream.read documentation, however the result of InputStreamSubscriber.read(byte[8192] 0, 0) is -1 which indicates that there is no more data to read.

This means that ResponseInputStream.readAllBytes will only ever return 8291 bytes of the ResponseInputStream.

Expected Behavior

ResponseInputStream.readAllBytes actually reads all bytes

Current Behavior

Only 8192 bytes are read

Reproduction Steps

var downloadRequestBuilder =
          DownloadRequest.builder()
              .getObjectRequest(request -> request.bucket("<SOME BUCKET>")
                  .key("<SOME KEY>"))
              .responseTransformer(AsyncResponseTransformer.toBlockingInputStream());

var downloadRequest = downloadRequestBuilder.build();
var download = s3TransferManager.download(downloadRequest);
var response =  download.completionFuture().join();

var byteArray = response.result().getAllBytes();

byteArray will only be 8192 bytes long

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.20.94

JDK version used

openjdk 17.0.7 2023-04-18 LTS OpenJDK Runtime Environment Zulu17.42+19-CA (build 17.0.7+7-LTS) OpenJDK 64-Bit Server VM Zulu17.42+19-CA (build 17.0.7+7-LTS, mixed mode, sharing)

Operating System and version

macOS 13.5

[faucct]

Would be fixed by this – #4258

[debora-ito]

@mgrundie-r7 thank you for the explanation, the description makes sense. I'm wondering if this has a reason behind or it was an oversight. We'll review the PR.

[debora-ito]

@faucct can you add tests to the PR?

[faucct]

I have tried to add it today, but I don't know how to make sure it gets skipped on Java 8, as there is no such method there yet:

    @EnabledForJreRange(min = JRE.JAVA_9)
    @Test
    public void readAllBytesShouldReceiveNoPrematureEOS() throws IOException {
        publisher.subscribe(subscriber);
        publisher.send(byteBufferOfLength(2 << 14));
        publisher.complete();
        assertThat(subscriber.readAllBytes()).isEqualTo(new byte[2 << 14]);
    }

Please, add it for me if you know how to do this.

[debora-ito]

Please ask @dagnir in the PR #4258, let's keep all PR things in one place.

[faucct]

Okay, though you’ve started writing here first. :)