8000 HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) by melissayou · Pull Request #5097 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) #5097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Nov 5, 2022
Merged

HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) #5097

merged 4 commits into from
Nov 5, 2022

Conversation

melissayou
Copy link
Contributor
@melissayou melissayou commented Nov 2, 2022 •
edited
Loading

Description of PR

To port the change from trunk to branch-3.3. It's a clean cherry pick of commit ccfa072.

How was this patch tested?

mvn clean install -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip builds successfully

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@virajjasani
HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…
2ab4794
8000 
…#3241)

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
ashutoshcipher
ashutoshcipher suggested changes Nov 2, 2022
View reviewed changes
Copy link
Contributor
@ashutoshcipher ashutoshcipher left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @melissayou for your contribution. I can see that you have another old PR for the same issue. #5067

Good to close one of them.

@virajith virajith self-requested a review November 2, 2022 00:18
@melissayou
Copy link
Contributor Author

I can see that you have another old PR for the same issue. #5067

Yes @ashutoshcipher, we are considering splitting up the original 5067 one to smaller PRs so that's why I am creating new ones. I still want to keep 5067 open until this is resolved for my own reference.

@ashutoshcipher
Copy link
Contributor

Thanks @melissayou. Incase you are planning to split it into multiple PRs. I think in that we can create a separate Uber JIRA and create multiple JIRAs under it according to split you are planning. It will make it easy to understand the split plan and reviewing each split.

Any suggestions on this @virajith ?

@melissayou melissayou changed the title HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241) HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) Nov 2, 2022
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 9m 56s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.3 Compile Tests _
+0 🆗 mvndep 15m 9s Maven dependency ordering for branch
+1 💚 mvninstall 27m 13s branch-3.3 passed
+1 💚 compile 18m 43s branch-3.3 passed
+1 💚 checkstyle 3m 2s branch-3.3 passed
+1 💚 mvnsite 21m 2s branch-3.3 passed
+1 💚 javadoc 7m 0s branch-3.3 passed
+0 🆗 spotbugs 0m 17s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 58m 55s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for patch
+1 💚 mvninstall 30m 28s the patch passed
+1 💚 compile 18m 12s the patch passed
-1 ❌ javac 18m 12s /results-compile-javac-root.txt root generated 10 new + 1862 unchanged - 0 fixed = 1872 total (was 1862)
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 2m 50s /results-checkstyle-root.txt root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚 mvnsite 20m 41s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 javadoc 6m 47s the patch passed
+0 🆗 spotbugs 0m 15s hadoop-project has no data from spotbugs
+1 💚 shadedclient 59m 36s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 719m 7s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 23s The patch does not generate ASF License warnings.
1051m 40s
Reason Tests
Failed junit tests hadoop.yarn.applications.distributedshell.TestDSTimelineV20
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoCreatedQueuePreemption
hadoop.yarn.sls.appmaster.TestAMSimulator
hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized
hadoop.hdfs.server.balancer.TestBalancer
hadoop.hdfs.tools.TestDebugAdmin
hadoop.hdfs.server.datanode.TestBPOfferService
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/1/artifact/out/Dockerfile
GITHUB PR #5097
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname Linux 0d30d3344d1b 4.15.0-192-generic #203-Ubuntu SMP Wed Aug 10 17:40:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / 2ab4794
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/1/testReport/
Max. process+thread count 2157 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2 shellcheck=0.4.6
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.3 Compile Tests _
+0 🆗 mvndep 14m 43s Maven dependency ordering for branch
+1 💚 mvninstall 27m 10s branch-3.3 passed
+1 💚 compile 19m 2s branch-3.3 passed
+1 💚 checkstyle 3m 14s branch-3.3 passed
+1 💚 mvnsite 21m 22s branch-3.3 passed
+1 💚 javadoc 7m 6s branch-3.3 passed
+0 🆗 spotbugs 0m 24s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 59m 54s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 36s Maven dependency ordering for patch
+1 💚 mvninstall 31m 36s the patch passed
+1 💚 compile 18m 35s the patch passed
-1 ❌ javac 18m 35s /results-compile-javac-root.txt root generated 10 new + 1863 unchanged - 0 fixed = 1873 total (was 1863)
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 3m 9s /results-checkstyle-root.txt root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚 mvnsite 21m 2s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 javadoc 6m 54s the patch passed
+0 🆗 spotbugs 0m 23s hadoop-project has no data from spotbugs
+1 💚 shadedclient 59m 55s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 715m 47s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 53s The patch does not generate ASF License warnings.
1046m 36s
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
hadoop.hdfs.tools.TestDFSZKFailoverController
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
hadoop.hdfs.web.TestWebHDFS
hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/2/artifact/out/Dockerfile
GITHUB PR #5097
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname Linux 45a09da06520 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / 44810d6
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/2/testReport/
Max. process+thread count 2300 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/2/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2 shellcheck=0.4.6
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@virajith
Copy link
Contributor
virajith commented Nov 3, 2022

Yes, that makes sense @ashutoshcipher. @melissayou created https://issues.apache.org/jira/browse/HADOOP-18518 as the parent JIRA for all backports related to upgrading these libraries.

@virajith
Copy link
Contributor
virajith commented Nov 3, 2022
8000

The change looks good to me @melissayou. Can you fix the checkstyle? After that, I am good to merge this as the javac deprecations will be fixed by the other tickets under HADOOP-18518

@slfan1989
Copy link
Contributor

@melissayou Thank you very much for your contribution, LGTM +1.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 59s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.3 Compile Tests _
+0 🆗 mvndep 0m 38s Maven dependency ordering for branch
+1 💚 mvninstall 25m 9s branch-3.3 passed
+1 💚 compile 19m 12s branch-3.3 passed
+1 💚 checkstyle 3m 7s branch-3.3 passed
+1 💚 mvnsite 21m 4s branch-3.3 passed
+1 💚 javadoc 6m 56s branch-3.3 passed
+0 🆗 spotbugs 0m 22s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 59m 54s branch has no errors when building and testing our client artifacts.
_ Patch Compi 8000 le Tests _
+0 🆗 mvndep 0m 25s Maven dependency ordering for patch
+1 💚 mvninstall 31m 56s the patch passed
+1 💚 compile 18m 36s the patch passed
-1 ❌ javac 18m 36s /results-compile-javac-root.txt root generated 10 new + 1863 unchanged - 0 fixed = 1873 total (was 1863)
-1 ❌ blanks 0m 0s /blanks-eol.txt The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 3m 6s /results-checkstyle-root.txt root: The patch generated 64 new + 24 unchanged - 334 fixed = 88 total (was 358)
+1 💚 mvnsite 21m 5s the patch passed
+1 💚 shellcheck 0m 1s No new issues.
+1 💚 javadoc 6m 56s the patch passed
+0 🆗 spotbugs 0m 23s hadoop-project has no data from spotbugs
+1 💚 shadedclient 60m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 734m 0s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 58s The patch does not generate ASF License warnings.
1048m 0s
Reason Tests
Failed junit tests hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
hadoop.hdfs.server.datanode.TestBPOfferService
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/4/artifact/out/Dockerfile
GITHUB PR #5097
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname Linux 78f877bbc7ea 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / c23620e
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/4/testReport/
Max. process+thread count 2279 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5097/4/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2 shellcheck=0.4.6
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@virajith
Copy link
Contributor
virajith commented Nov 5, 2022

The checkstyle failures exist in the original patch as well - fixing all those will change the patch substantially and it'll no longer be a clean cherrypick. In order to keep this as a simple cherrypick, I think we can ignore these. Will be merging PR.

@virajith virajith merged commit 853ffb5 into apache:branch-3.3 Nov 5, 2022
NyteKnight pushed a commit to NyteKnight/hadoop that referenced this pull request Jun 25, 2024
@virajjasani
HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper t…
b5aad20 
…o 3.6.3 and Curator to 5.2.0) (apache#5097) (apache#24)

* HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache#3241)

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Co-authored-by: Viraj Jasani <vjasani@apache.org>
Co-authored-by: Melissa You <myou@linkedin.com>

Co-authored-by: Viraj Jasani <vjasani@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashutoshcipher ashutoshcipher ashutoshcipher requested changes

@virajith virajith virajith approved these changes

@slfan1989 slfan1989 slfan1989 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@melissayou @ashutoshcipher @hadoop-yetus @virajith @slfan1989 @virajjasani
0