netty: Swap to UniformStreamByteDistributor

[forthehell]

Is your feature request related to a problem?

We send the asyncUnaryCall in line in the same thread and with only one channel ， we find the rpc call arrived in server is out of order。

Describe the solution you'd like

We find the reason is the WeightedFairQueueByteDistributor 。 And theUniformStreamByteDistributor can more likely do the job， but there is no configuration to switch between those ByteDistributors。 can we add the configuration about it？

if(FAIR_Distributor){
  WeightedFairQueueByteDistributor dist = new WeightedFairQueueByteDistributor(connection);
  dist.allocationQuantum(16 * 1024); // Make benchmarks fast again.
  streamByteDistributor = dist;
}else{
  UniformStreamByteDistributor uniformStreamByteDistributor =  new UniformStreamByteDistributor(connection);
  uniformStreamByteDistributor.minAllocationChunk(16 * 1024);
  streamByteDistributor = uniformStreamByteDistributor;
}

Describe alternatives you've considered

Additional context

[ejona86]

If you start two asynchronous RPCs, there's no guarantees about their orders. The distributor doesn't even guarantee you what you're wanting, as the message size can change the order.

Since priorities were deprecated in RFC 9113, it might be reasonable to just change to UniformStreamByteDistributor unconditionally. We should also look into what's happening with Netty in this space (esp Netty 4.2), as they would have changed their default in the http2 builder, that grpc doesn't use.

So using the uniform distributor can be fair. But it is very bad to do it for your purpose. There are no ordering guarantees unless you get some response from the server before sending another RPC/message.

[forthehell]

If you start two asynchronous RPCs, there's no guarantees about their orders. The distributor doesn't even guarantee you what you're wanting, as the message size can change the order.

Since priorities were deprecated in RFC 9113, it might be reasonable to just change to UniformStreamByteDistributor unconditionally. We should also look into what's happening with Netty in this space (esp Netty 4.2), as they would have changed their default in the http2 builder, that grpc doesn't use.

So using the uniform distributor can be fair. But it is very bad to do it for your purpose. There are no ordering guarantees unless you get some response from the server before sending another RPC/message.

I wonder why the message size can change the order ? asynchronous RPCs dataFrame will be sent to the WriterQueue which belonged to the save channel。if flush the WriterQueue command ，they will be comsumed in order 。 if the distributor is fair， then they will be sent in order too。 I mean no ordering guarantees is the design concept， but if using the uniform distributor， it is very bad to do it for my purpose， but actually，it can do it ？

[ejona86]

I wonder why the message size can change the order ?

If the first message is bigger then the second message, then the fair distributor could finish sending the second message first. Whether this actually happens is dependent on the quantum size (16 KiB on that line), but that configuration is highly internal.

Even if they are sent in-order, the server's executor could end up running the second one "first," due to both running concurrently (so which is "first" is dependent on your application logic) or kernel thread scheduling.

Then there's other cases like, "the two RPCs may not end up on the same connection," because of GOAWAY.

[ejona86]

This issue will look at swapping to UniformStreamByteDistributor, since priorities are deprecated. We should first understand what is happening in Netty, especially Netty 4.2. But our decision can be separate from theirs.

(The goal of this change is not to preserve the order of RPCs. That can be a side-effect, but not the goal)

[shivaspeaks]

The history to note here is, we started with WeightedFairQueueByteDistributor which was also the default flow controller with netty. But then after a while we avoided the default and started using UniformStreamByteDistributor here, #1456 - some discussion happened in netty around headers could go out of context, netty/netty#4758.
This change was undone somewhere in between and default was applied. I couldn't find that PR.

And then after that we have #3038 that talks about increasing message quantum with WeightedFairQueueByteDistributor. Why didn't we use Uniform flow control even when its QPS was higher (Ref: #3038 (comment))?

[kannanjgithub]

We can learn the history in due course, but the more important work to start with is to understand whether Netty's HTTP2 default builder has switched to the uniform distributor as mentioned in this comment.

[shivaspeaks]

No, netty's default builder still uses WeightedFairQueueByteDistributor.

[ejona86]

The primary http2 builder in Netty is Http2MultiplexCodecBuilder. There's also Http2FrameCodecBuilder, which we would be more likely to use, but is rarely used by Netty users.

In those code flows, the StreamByteDistribution is not being passed to the constructor, so the constructor Shiva linked to is relevant.