Skip to content

Make producer's event channel size configurable #87

@agis

Description

@agis

In rafka, we tend to create a producer per client connection. We also sometimes create short-lived producers, that produce a single message and then are destroyed.

confluent-kafka-go sets the events channel buffer size for each producer to a fixed 1000000 (1M) and doesn't provide a way for the client to configure this:

p.events = make(chan Event, 1000000)

This, combined with golang/go#16930 can be problematic in the setup mentioned above. In our case it results in about 2GB of RSS not being handled back to the OS. The problem could be mitigated, although not solved, if the client could control the channel size.

We could for example reuse the go.produce.channel.size value when creating the p.events channel.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions