If cluster id is null and observation is enabled, KafkaTemplate tries to retrieve it on each produce

[m-burst]

In what version(s) of Spring for Apache Kafka are you seeing this issue?

3.0.11

Describe the bug

If observation is enabled on a KafkaTemplate, whenever it produces a message, it tries to retrieve the cluster ID from the Kafka cluster.
Normally the cluster ID is retrieved once and then cached, but if the cluster does not have a cluster ID configured, the cached value remains null, and the request is made again on each next produce.
This is highly inefficient, because an AdminClient is created every time for this purpose.

See KafkaTemplate#clusterId(), which in turn calls KafkaAdmin#clusterId()

To Reproduce

1. Run a Kafka cluster without cluster ID configured
2. KafkaTemplate.setObservationEnabled(true)
3. Produce multiple messages
4. Observe the application logs: an AdminClient is being created for each produce attempt, and its config is printed

Expected behavior

KafkaTemplate can distinguish between the situations where it did not yet retrieve the cluster ID and when it is not configured on the cluster.

Sample

No sample yet, but I can prepare one if needed.

[garyrussell]

How do you create a cluster with no id?

In my experience, there is always an ID, e.g. 2zFC3alOTS-6ghmOTZmA6w.

[m-burst]

I have been using Redpanda v21.11 as a drop-in replacement for Kafka for local development, and it has no cluster id by default.

After your reply I have tested this with a newer version of Redpanda, and generates a random id by default.

Not sure whether this is worth fixing, leaving this up to you

[garyrussell]

Thanks; we will just set some value when if the broker returns null.