Per topic config

[ecksun]

Description

I would like to be able to specify auto.offset.reset per topic.

I have some topics where its fine to miss messages but are huge (and I thus don't want to start over) and some topics where exactly once semantics are very important.

It seems like librdkafka supports this, however I cannot find any good documentation on how to use it from confluent-kafka-python.

Googling only really turns up this issue relating to the go client

How to reproduce

N/A

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): N/A
• Apache Kafka broker version: N/A
• Client configuration: Yes, that is the question
• Operating system: N/A
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue

[edenhill]

Per-topic configuration is not currently exposed in the Python client, but your interest in this feature is duly noted.

As a workaround you can implement your own on_assign callback and read committed offsets for the given partition set, if any partition does not have a committed offset you set the start offset to either END or BEGINNING according to your policy.

Please vote on this feature by up-thumbing this comment