Skip to content

Distinct concurrency for retry and DLT topic containers #2443

@garyrussell

Description

@garyrussell

Discussed in #2434

Originally posted by jgslima October 11, 2022
I am struggling to understand how to achieve a configuration where I can have the listener container for the retry and DLT topics with a concurrency different from the container of the main topic (the retry and DLT topics do not need the same amount of threads/consumers).

I have two beans of ConcurrentKafkaListenerContainerFactory<?, ?>:

  • one with the name kafkaListenerContainerFactory, to be used as the main factory for the main containers.
  • another one, dedicated to the retry/DLT containers, configured with the desired concurrency for them.

Then, I already tried to force the use of the second factory for the retry/DTL containers with these 2 ways:

  • specifying the factory name in the listenerContainerFactory attribute of the @RetryableTopic annotation.
  • forcing the name of this factory to be RetryTopicBeanNames.DEFAULT_LISTENER_CONTAINER_FACTORY_BEAN_NAME.

But in the end, I always end up having, for the retry and DLT topic, an amount of containers/threads equal to the concurrency of the main listener (that is, the concurrency declared in the @KafkaListener annotation).

I tried to understand and debug ListenerContainerFactoryResolver, at some point it seems to resolve to the factory with the customized concurrency, but in the end for some reason the amount of containers/threads are egual to the main listener.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions