Configurable chunking/batching in PulsarTemplate

[onobc]

Batching configuration

Batching is enabled by default in Pulsar. This can be configured globally by setting spring.pulsar.producer.batching-enabled. We should provide a simple mechanism to enable/disable/configure batching on a per-send-operation basis.

Chunking configuration

Chunking is disabled by default in Pulsar. This can be configured globally by setting spring.pulsar.producer.chunking-enabled. We should provide a simple mechanism to enable/disable/configure chunking on a per-send-operation basis.

NOTE: When chunking is enabled batch must be disabled. However, we will delegate to underlying Pulsar producer validation handling this. It does an extensive job of client validation.

[onobc]

Batch/Chunking impl note

Because these are producer level configuration options we will need to create the concept of ProducerBuilderCustomizer and then pass them down upon creation. Similar to what we are doing w/ the TypedMessageBuilderCustomizer currently.

The fluent template will lend itself well to allowing the users to specify the option(s). Will need a bit of work to pass that down into the DefaultProducerFactory.

We will need to decide whether to expose the entire builder or just the fine grained levers we want to.

Option1: Expose the entire builder

public SendMessageSpec<T> producerCustomizedWith(ProdcuerBuilderCustomizer<T> customizer) {
	producerCustomizers.add(customizer);
	return this;
}

Option 2: Expose only those options we want

public SendMessageSpec<T> withBatching(boolean enabled) {
	producerCustomizers.add((producerBuilder -> producerBuilder.enableBatching(enabled)));
	return this;
}

public SendMessageSpec<T> withChunking(boolean enabled) {
	producerCustomizers.add((producerBuilder -> producerBuilder.enableChunking(enabled)));
	return this;
}

Tradeoffs include:

• Option1
  • (➕) full power to users / less maintenance feature requests for us to add
  • (➖) more possible for user to shoot themselves in the foot w/ too much power
• Option2
  • (➕) less power to users / more maintenance feature requests for us to add
  • (➖) less possible for user to shoot themselves in the foot w/ too much power

@sobychacko @alpreu what option should we go w/? I am in favor of option1 because I feel the validation of the internal pulsar producer is fairly extensive thus minimizing the risk of them "shooting themselves in the foot". Wdyt?

[onobc]

My feel would also be to go with option 1

Originally posted by @alpreu in #16 (comment)