Skip to content

Distinct concurrency for retry and DLT topic containers #2443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
garyrussell opened this issue Oct 18, 2022 Discussed in #2434 · 0 comments · Fixed by #2444
Closed

Distinct concurrency for retry and DLT topic containers #2443

garyrussell opened this issue Oct 18, 2022 Discussed in #2434 · 0 comments · Fixed by #2444
Assignees
Milestone

Comments

@garyrussell
Copy link
Contributor

Discussed in #2434

Originally posted by jgslima October 11, 2022
I am struggling to understand how to achieve a configuration where I can have the listener container for the retry and DLT topics with a concurrency different from the container of the main topic (the retry and DLT topics do not need the same amount of threads/consumers).

I have two beans of ConcurrentKafkaListenerContainerFactory<?, ?>:

  • one with the name kafkaListenerContainerFactory, to be used as the main factory for the main containers.
  • another one, dedicated to the retry/DLT containers, configured with the desired concurrency for them.

Then, I already tried to force the use of the second factory for the retry/DTL containers with these 2 ways:

  • specifying the factory name in the listenerContainerFactory attribute of the @RetryableTopic annotation.
  • forcing the name of this factory to be RetryTopicBeanNames.DEFAULT_LISTENER_CONTAINER_FACTORY_BEAN_NAME.

But in the end, I always end up having, for the retry and DLT topic, an amount of containers/threads equal to the concurrency of the main listener (that is, the concurrency declared in the @KafkaListener annotation).

I tried to understand and debug ListenerContainerFactoryResolver, at some point it seems to resolve to the factory with the customized concurrency, but in the end for some reason the amount of containers/threads are egual to the main listener.

@garyrussell garyrussell added this to the 3.0.0-RC2 milestone Oct 18, 2022
@garyrussell garyrussell self-assigned this Oct 18, 2022
garyrussell added a commit to garyrussell/spring-kafka that referenced this issue Oct 18, 2022
Resolves spring-projects#2443

Optionally configure a different concurrency for retry containers.
artembilan pushed a commit that referenced this issue Oct 21, 2022
Resolves #2443

Optionally configure a different concurrency for retry containers.

* Add Javadocs to RetryTopicConfigurationBuilder.

* Javadoc polishing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant