Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

What changes were proposed in this pull request?

This PR replaces the deprecated KafkaConsumer.poll(long) with KafkaConsumer.poll(Duration). The old one is deprecated at Kafka 2.0.0.

Why are the changes needed?

This will use the latest API.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Jenkins.

package org.apache.spark.sql.kafka010

import java.{util => ju}
import java.time.{Duration => jDuration}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will avoid conflicts with import scala.concurrent.duration.Duration at line 27.

@HeartSaVioR
Copy link
Contributor

Sorry, but we tried it out and that cannot be simple replacement.

Please refer https://issues.apache.org/jira/browse/SPARK-28367 and #25135

We were waiting for Kafka 2.4.0 to deal with poll(0) replacement: Kafka 2.4.0 is now available, but if we have time to address, I'd like to give @gaborgsomogyi a chance to work for this, as he has been waiting for Kafka 2.4.0. He'll be back on next month.

@dongjoon-hyun
Copy link
Member Author

Oh, I didn't notice that effort. Let me take a look. Thanks, @HeartSaVioR .

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115585/
Test FAILed.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-30311 branch December 19, 2019 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants