After calling close, consumers issue periodic metadata requests

[aaronhays]

If you create a consumer, connect, read some data, and then close but don't destroy every reference to the consumer, it will continue issuing metadata requests every five minutes.

I'm assuming this is because you are calling rd_kafka_consumer_close in the python close() method, but not calling rd_kafka_destroy until the consumer object is deallocated. It seems like once you call close(), the Consumer instance cannot be used for anything - is there a reason not to call rd_kafka_destroy in close()?

[edenhill]

Your analysis is correct.
close() "only" closes the high-level consumer functionality (namely leaving the group, committing offsets, etc), but the underlying librdkafka consumer handle remains functional for legacy consumer and basic operations (such as metadata requests!).

Now, there's probably no real use to close the consumer and then do something else on the same handle, so calling rd_kafka_destroy() from close() in the Python API sounds like a good idea.

[edenhill]

Fixed on master.