A new feature as a workaround hack to unavailable offset support in Arrow Java

[viirya]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

For offset-ed offset buffer of variable-sized binary array, the valid FFI output is to record the offset buffer's offset into FFI's offset field and expose the beginning of the buffer as the pointer. So the consumer can calculate correctly the length of data buffer.

However, due to Arrow Java's issue: apache/arrow-java#74, Arrow Java does not support offset field in the FFI output. It causes no sliced binary array can be passed between arrow-rs and Arrow Java.

To workaround this issue, we have to enforce offset-ed offset buffer to be exposed at the sliced buffer's beginning. When calculating the length of data buffer, we assume the first element of the offset buffer is always 0 (this is how Arrow Java and arrow-rs do).

This is a hacky solution and should be removed once Arrow Java supports offset field in the FFI output.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[alamb]

label_issue.py automatically added labels {'arrow'} from apache/arrow#5964