DefaultKafkaProducerFactory - add threadLocalProducer option

[bekker]

Affects Version(s): all version

DefaultKafkaProducerFactory currently produces a singleton Producer (#8), because Producer is thread-safe and faster in that way.

The producer is thread safe and sharing a single producer instance across threads will generally be faster than having multiple instances. - from KafkaProducer docs

Thus KafkaTemplate with DefaultKafkaProducerFactory shares one Producer across all threads.

But when autoFlush set to true, KafkaTemplate tries to flush shared Producer for every send request.

When a fail or a delay occurs on Kafka cluster and a send request takes a long time, every other send requests have to wait for it while they don't have to.

One delay may cause unnecessary full-stop on whole application.

I think DefaultKafkaProducerFactory should always create new object - not singleton - so that KafkaTemplate be able to choose to use thread-local Producer or not. But in that case there would be huge implications... any suggestions?

[garyrussell]

• Add an option the DKPF to maintain producers in a ThreadLocal instead of a singleton.
• add closeProducer to the DKPF so clients can physically close the producer and clean up the TL when no longer needed