Different per-topic timeout configuration

[swehner]

Description

I'm looking to use a single Kafka producer for two slightly differnt use cases/message types. One that is just fire-and-forget and that will simply enqueue a message without wait for any acknowledgment.
For the second use case I'd like to make sure that the message was written to broker before returning a response - similar to what's described in #1174

It seems that the way to achieve this would be to set a low message timeout and wait for poll() to acknowledge message delivery via the callback.

Having a low global message timeout wouldn't be very good for the fire-and-forget use cause, so it seems that the best option would be to use the per-topic configuration.
Was there any progress on implementing this since #328 (comment) ? It looks like none of the APIs are exposed through the python client.

If there's still no way to provide per-topic config I'd be open to work on it and make a pull request, but would like to understand what the preferred approach is. maybe exposing the topic creation methods on the produce with a config dict would be enough?

Thanks in advance

Checklist

Please provide the following information:

• confluent-kafka-python==2.20 librdkafka==2.2.0
• Apache Kafka broker version: 3.4.0
• Client configuration: {...}
• Operating system:
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue

[swehner]

I've created a pull-request for exposing the topic_new to be able to set a custom configuration here: #1695

Let me know if that approach works for you