HADOOP-15984. Update jersey from 1.19 to 2.x. #7019
Merged
+32,897
−26,930
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as outdated.
This comment was marked as outdated.
|
The code for |
slfan1989
force-pushed
the
HADOOP-15984
branch
from
3b35b60 to
820dc89
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
I am continuing to follow up on the remaining issues, including sputbug, checkstyle, |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
slfan1989
force-pushed
the
HADOOP-15984
branch
from
7323c92 to
6971140
Compare
|
💔 -1 overall
This message was automatically generated. |
|
Most of the issues have been resolved, but the overall renovation time has taken longer than expected. There are still some problems with checkstyle, spotbugs, and shadeclient that are being addressed. I hope to use the weekend to resolve these issues. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
slfan1989
force-pushed
the
HADOOP-15984
branch
from
0e787f9 to
eb52efc
Compare
|
💔 -1 overall
This message was automatically generated. |
slfan1989
changed the title
|
💔 -1 overall
This message was automatically generated. |
|
The compilation report contains the following error: I will add the following dependencies to the relevant module. |
|
I think we should merge this as is and make outstanding issues followups. We will see problems, just for different deployments/test runs, so let's just encourage testing with the goal of finding those corner cases. We need to do this for java17, so let's get it in and see what those corner cases are |
|
💔 -1 overall
This message was automatically generated. |
4 tasks
Many thanks to @steveloughran for the comment! Here’s a summary of the current situation: all compilations have passed. The reported unit test failures include one that often times out (TestDecommission, which frequently appears in other PRs as well) and another that might be an occasional runtime failure (TestTimelineAuthFilterForV2), but it passes locally for me. I have checked the checkstyle and imports for all classes, and they should meet the expected standards. A special thanks to @ayushtkn for providing a Yetus package that has been helpful for continuous debugging. Before merging this PR, I will roll back the changes to the |
Hexiaoqiao
reviewed
Jan 15, 2025
LICENSE-binary
Outdated
| com.sun.jersey:jersey-guice:1.19.4 | ||
| com.sun.jersey:jersey-server:1.19.4 | ||
| com.sun.jersey:jersey-servlet:1.19.4 | ||
| org.glassfish.jersey.core:jersey-common:2.46 |
There was a problem hiding this comment.
It is OK to located this package to Apache License or MIT if it is dual licensed IMO, if we could not make sure, maybe refer Spark https://github.com/apache/spark/blob/master/LICENSE-binary and write it under Apache License will not be wrong. Thanks.
dev-support/Jenkinsfile
Outdated
| @@ -71,7 +71,7 @@ pipeline { | |||
| checkout([ | |||
| $class: 'GitSCM', | |||
| branches: [[name: "${env.YETUS_VERSION}"]], | |||
| userRemoteConfigs: [[ url: 'https://github.com/apache/yetus.git']]] | |||
| userRemoteConfigs: [[ url: 'https://github.com/ayushtkn/yetus.git']]] | |||
There was a problem hiding this comment.
It is better to link Hadoop repo rather than Ayushtkn's personal repo.
There was a problem hiding this comment.
@Hexiaoqiao Thank you very much for your message! I have carefully checked the agreement for the new package we introduced, as follows:
Apache License Version 2.0
org.glassfish.jersey.core:jersey-common:2.46
org.glassfish.jersey.core:jersey-server:2.46
org.glassfish.jersey.inject:jersey-hk2:2.46
org.glassfish.jersey.core:jersey-client:2.46
org.glassfish.jersey.test-framework:jersey-test-framework-core:2.46
org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-grizzly2:2.46
org.glassfish.jersey.containers:jersey-container-servlet:2.46
org.glassfish.jersey.containers:jersey-container-servlet-core:2.46
org.glassfish.jersey.media:jersey-media-json-jettison:2.46
org.glassfish.jersey.media:jersey-media-jaxb:2.46
net.jodah:failsafe:2.4.4
Eclipse Distribution License (EDL) 1.0
jakarta.xml.bind:jakarta.xml.bind-api:2.3.3
Eclipse Public License (EPL) 2.0
jakarta.ws.rs-api:jakarta.ws.rs-api:2.1.6
jakarta.servlet.jsp:jakarta.servlet.jsp-api:2.3.6
jakarta.servlet:jakarta.servlet-api:4.0.4
cc: @pjfanning
steveloughran
approved these changes
Jan 16, 2025
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@steveloughran @Hexiaoqiao @pjfanning Thank you all for reviewing this PR! We have compiled it multiple times, and currently, the failing unit tests seem unrelated to our changes. Thanks to @pjfanning for the suggestions—they have been really helpful. Based on the current progress, I plan to merge this PR first and then assess if there are any other improvements we can submit in future PRs. Additionally, I will revert the changes to |
|
💔 -1 overall
This message was automatically generated. |
|
Thanks all for your great works. make sense to. +1 from my side. |
Hexiaoqiao
approved these changes
Jan 17, 2025
|
@Hexiaoqiao @steveloughran Thank you for approving this PR! If any issues arise in the future, I will continue to follow up. However, I remain optimistic since we haven't changed the core logic, and the primary changes are related to testing. I will also keep optimizing the code to prepare for a future upgrade to Jersey 3, and I already have some ideas for that. I plan to merge this PR in 2 working days. I will add the following members as co-authors in the commit to express my gratitude. @aajisaka @@virajjasani These two members provided the initial version of HADOOP-15984. At the same time, I hope our JDK 17 upgrade goes smoothly. After this PR, we plan to submit a large-scale PR to upgrade from JUnit4 to JUnit5. We have already prepared for this locally. |
|
I plan to merge this PR into the trunk branch and monitor the community build emails and test reports from the past two days. If the reports look good, I will backport the changes to branch-3.4 to support JDK 17 compatibility for the hadoop-3.4.2 release. A big thanks again to all the members who have reviewed and provided assistance with this PR. |
|
I believe the overall upgrade is currently in line with expectations. I have reviewed the overall build report, and there are no issues with JDK 8. However, with JDK 11, there is a reported issue regarding the missing |
|
Huge thanks @slfan1989 and all the reviewers. |
5 tasks
@aajisaka Thank you for your message! You helped take the first step in this PR, as the code merged into the community came from your submission, especially the HDFS part, which is crucial. Thanks again to the members who have been following this JIRA. |
|
think the junit changes broke a lot of testing running; for now I'm rebasing my prs back onto the commit before that. I don't want to revert this (it's big!), but I need to understand why junit 4 tests would stop running and what is needed to turn them back on |
|
@steveloughran I am prioritizing the investigation of this issue. If no other solutions are found, we will roll back this PR. I believe the problem might be related to the changes I made to the JUnit dependencies in the POM file. I'll check and try to submit a fix PR. I won't keep everyone waiting for too long. I've paused other tasks and am prioritizing this issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
JIRA: HADOOP-15984. Update jersey from 1.19 to 2.x.
Many members of the community have tried to help upgrade this component, and there are numerous PRs available for reference. I have also reviewed a lot of the code they submitted. I would like to express my sincere gratitude to them.
Differences from previous approaches
My approach differs from previous upgrade PRs in the following two main ways:
I did not insist on binding
GuiceandJerseytogether. InJersey 1.x, the official Jersey integration with Guice made dependency injection easier. However, inJersey 2.x, this integration has become quite weak and presents many issues during the integration process.I allowed
GuiceandJersey2to manage their own injected objects independently. Based on practical experience, this approach is feasible and does not cause conflicts.Key points of the upgrade include
We are attempting to upgrade Jersey to version 2.x to enable Hadoop to support higher JDK versions in the future. During the upgrade process, the following modifications were made:
For version selection, I chose Jersey
2.46.Jersey 2.x is Java EE / Jakarta EE 8 / JAX-RS 2.1 / Jakarta REST 2.1 compatible.
Jakartapackages to replace the originalJavaxpackages.javax.xml.bind -> jakarta.xml.bind
javax.activation -> jakarta.validation
javax.servlet -> jakarta.servlet-api
1.9.4, including the related exclusion tags in the POM.How was this patch tested?
Junit Test.
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?