With KEY ordering, option to combine queues from different partitions or topics

[astubbs]

In key mode, if records of a given key exist in

• multiple partitions of a topic (and those partitions are assigned to me)
• or when subscribing to multiple topics which have matching keys, have an option to have these either combined into a single queue, or kept separate.

If combined, option to choose how records are ordered relatively - e.g. via time stamp extractor, offset or publish time stamp.

Related:

• Subscribing to two or more topics with KEY ordering, results in messages of the same Key never being processed #184

[phanivankadari]

We are looking at similar use case where we have multiple topics (for prioritization and fairness usecases) and we wanted key grouping across all topics related to a microservice (wildcard topic or list of topics specified). I suppose the current key grouping is supported to work for messages for a given key at a single partition level - is this assumption correct?

I see the PR is closed as stale but did not have much comments if had any other issues. Any updates on the feasibility of this feature request?

[JorgenRingen]

Any plans to implement this i the near future? :-)

[rkolesnev]

Not really - it would be quite niche feature with a lot of caveats - main one - there is really no easy and bulletproof way to guarantee deterministic ordering of messages with same key when consumed from different topics across rebalances, subscription changes - even poll batching / sizing may play a role in how messages are read from the topics and ultimately sequenced.