Missing API: zero-copy wrapping of ByteArray with Source.

[SPC-code]

There are several cases in binary parsing, where one needs to wrap a ByteArray with an interface like Source and read from it sequentially. It should be possible by using a Buffer with a single Segment, but Segment API is private.

There should probably be two versions of the API: read/write and read-only.

[whyoleg]

I think, that wrapping ByteArray (or ByteBuffer) or any other unsafe API for working with Segment will be considered in scope of #135

[joffrey-bion]

I would also love to see this added, and I would add that an API for turning ByteString into a Source directly would also be very useful, because we cannot publicly access the ByteString's backing byte array from outside this library.

[fzhinkin]

@joffrey-bion

we cannot publicly access the ByteString's backing byte array from outside this library.

In fact, you can write something like:

@file:OptIn(UnsafeByteStringApi::class)
import kotlinx.io.bytestring.*
import kotlinx.io.bytestring.unsafe.*

val str = ByteString(1, 2, 3)

UnsafeByteStringOperations.withByteArrayUnsafe(str) { arr -> 
     println("There are ${arr.distinct().size} distinct bytes inside!")
}

[joffrey-bion]

Thanks for this! I wasn't aware, but that looks quite promising. I will check if this solves my problem and report back.

[fzhinkin]

There was an intention to avoid explicitly mentioning the unsafe API, so nobody would find it until that API is really needed. But it seems like we made it too hidden. :)

[joffrey-bion]

I put my thoughts in #259, because I don't want to pollute this issue any further (it's related but different)

[Chuckame]

After reading the #135 and #311 , I'm not sure, is there something around wrapping a ByteArray as a Source or a Buffer to enable zero-copy ? Sorry if I misread. It would add so much kotlin power for avro4k, allowing me to only provide an API using kotlinx-io Source and Sink, and the users are free to wrap a source from a ByteArray, ByteBuffer, a file, network, etc

[fzhinkin]

@Chuckame please check

kotlinx-io/core/common/src/unsafe/UnsafeBufferOperations.kt

Line 37 in fd49af5

public fun moveToTail(buffer: Buffer, bytes: ByteArray, startIndex: Int = 0, endIndex: Int = bytes.size) {

[Chuckame]

Thanks, so is this open issue still relevant ?