-
Notifications
You must be signed in to change notification settings - Fork 697
Description
In rafka, we tend to create a producer per client connection. We also sometimes create short-lived producers, that produce a single message and then are destroyed.
confluent-kafka-go sets the events channel buffer size for each producer to a fixed 1000000
(1M) and doesn't provide a way for the client to configure this:
confluent-kafka-go/kafka/producer.go
Line 319 in 9f5af42
p.events = make(chan Event, 1000000) |
This, combined with golang/go#16930 can be problematic in the setup mentioned above. In our case it results in about 2GB of RSS not being handled back to the OS. The problem could be mitigated, although not solved, if the client could control the channel size.
We could for example reuse the go.produce.channel.size
value when creating the p.events
channel.