GH-3814: Upgrade Apache Kafka client dependencies to version 4.0.0 #3815
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ersion 4.0.0 Fixes: spring-projects#3814 Issue link: spring-projects#3814 This commit upgrades the Apache Kafka client to 4.0.0 with the following changes: - Upgrade Kafka client dependencies to 4.0.0 in build.gradle - Remove ZooKeeper-based broker implementation as Kafka 4.0 fully transitions to KRaft mode - Delete EmbeddedKafkaZKBroker class and related tests - Remove ZooKeeper dependency - Refactor EmbeddedKafkaRule to use KRaft exclusively - Update KafkaClusterTestKit imports to use new packages - Update ConsumerRecords constructor calls to include the new required Map parameter - Add implementations for new Producer interface methods: - registerMetricForSubscription - unregisterMetricFromSubscription - Update KafkaStreamBrancher to use new split() and branch() methods - Remove deprecated partitioner classes from runtime hints - Remove deprecated sendOffsetsToTransaction method that used String consumerGroupId - Update BrokerAddress to use org.apache.kafka.server.network.BrokerEndPoint - Update DeserializationExceptionHandler to use new ErrorHandlerContext The commit also includes test modifications to address limitations with static port assignments in KRaft mode and adjustments to replication factors in tests. Signed-off-by: Soby Chacko <[email protected]>
artembilan
requested changes
Mar 21, 2025
There was a problem hiding this comment.
Just couple simple nit-picks.
This is really cool!
And change is not that bad: mostly tests adjustments because of new ctor for ConsumerRecords 😄
The removal of EmbeddedKafkaRule is a bit destructive. But who uses JUnit 4 these days!
So, or we revise it for KRaft or just bite a bullet.
The docs has to be changed as well.
That might be some effort, so consider to do that in the separate issue/PR.
Thanks
build.gradle
Outdated
| @@ -355,16 +355,14 @@ project ('spring-kafka-test') { | |||
| description = 'Spring Kafka Test Support' | |||
|
|
|||
| dependencies { | |||
| implementation "org.apache.logging.log4j:log4j-slf4j-impl:$log4jVersion" | |||
There was a problem hiding this comment.
Has to be api .
That's exactly how Java libraries are designed with Gradle.
| @@ -84,22 +88,37 @@ public KafkaStreamBrancher<K, V> defaultBranch(Consumer<? super KStream<K, V>> c | |||
|
|
|||
| /** | |||
| * Terminating method that builds branches on top of given {@code KStream}. | |||
| * Applies each predicate-consumer pair sequentially to create branches. | |||
| * If a default consumer exists, it will handle all records that don't match any predicates. | |||
| * | |||
There was a problem hiding this comment.
No blank lines in method Javadocs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #3814
Issue link: #3814
This commit upgrades the Apache Kafka client to 4.0.0 with the following changes:
Upgrade Kafka client dependencies to 4.0.0 in build.gradle
Remove ZooKeeper-based broker implementation as Kafka 4.0 fully transitions to KRaft mode
Update ConsumerRecords constructor calls to include the new required Map parameter
Add implementations for new Producer interface methods:
Update KafkaStreamBrancher to use new split() and branch() methods
Remove deprecated partitioner classes from runtime hints
Remove deprecated sendOffsetsToTransaction method that used String consumerGroupId
Update BrokerAddress to use org.apache.kafka.server.network.BrokerEndPoint
Update DeserializationExceptionHandler to use new ErrorHandlerContext
The commit also includes test modifications to address limitations with static port assignments in KRaft mode and adjustments to replication factors in tests.