Arrow fails with Netty 4.1.120.Final+

[wendigo]

Describe the bug, including details regarding any error messages, version, and platform.

Caused by: UnsupportedOperationException
	at EmptyByteBuf.memoryAddress(EmptyByteBuf.java:955)
	at DuplicatedByteBuf.memoryAddress(DuplicatedByteBuf.java:115)
	at UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:45)
	at PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:47)
	at NettyAllocationManager.<clinit>(NettyAllocationManager.java:54)
	... 128 more

[wendigo]

static final ArrowBuf EMPTY_BUFFER =
      new ArrowBuf(ReferenceManager.NO_OP, null, 0, NettyAllocationManager.EMPTY.memoryAddress());

this throws UnsupportedOperationException when memoryAddress() is called on EmptyByteBuf

[lidavidm]

Hmm, thanks. Hopefully this can be fixed upstream since trying to branch on every potential address check would be painful.

FWIW, we are tied to Unsafe/Netty until/unless someone can step up and rework things to work with the new native memory APIs (while figuring out how to preserve backwards compatibility somehow...)

[jbonofre]

Thanks for sharing. We should plan an effort about using new native memory APIs. I will take a cranck.

[lidavidm]

It sounds like the issue is just that we need to also require -Dio.netty.noUnsafe=false and/or --sun-misc-unsafe-memory-access=allow for users on JDK24 and above?

[wendigo]

@lidavidm correct, that works but this is not a long-term solution as the Unsafe will eventually go away

[jbonofre]

@lidavidm it should work. I'm fine to move forward like this, as soon as we put a TODO to use the new native APIs at some point 😄

[lidavidm]

We have a todo: #163

Let's use this issue to (1) test Java 24 in CI (2) update the documentation

[jbonofre]

I updated to Netty 4.2.1.Final and it seems to work. Can you please double check ?