8000 HADOOP-18088. Replace log4j 1.x with reload4j. by iwasakims · Pull Request #4151 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HADOOP-18088. Replace log4j 1.x with reload4j. #4151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 14, 2022
Merged

HADOOP-18088. Replace log4j 1.x with reload4j. #4151

merged 2 commits into from
Apr 14, 2022

Conversation

iwasakims
Copy link
Member

@iwasakims @jojochuang
HADOOP-18088. Replace log4j 1.x with reload4j.
f241a09 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
@iwasakims
Copy link
Member Author 
$ mvn clean install -DskipTests -Pnative -Pdist
$ find hadoop-dist/target/hadoop-2.10.2-SNAPSHOT -name '*log4j*.jar'
$ find hadoop-dist/target/hadoop-2.10.2-SNAPSHOT -name '*reload4j*.jar'
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/common/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/hdfs/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/slf4j-reload4j-1.7.36.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/slf4j-reload4j-1.7.36.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/yarn/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/mapreduce/lib/reload4j-1.2.18.3.jar
hadoop-dist/target/hadoop-2.10.2-SNAPSHOT/share/hadoop/tools/lib/reload4j-1.2.18.3.jar

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 8m 17s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-2.10 Compile Tests _
+0 🆗 mvndep 3m 52s Maven dependency ordering for branch
+1 💚 mvninstall 13m 48s branch-2.10 passed
+1 💚 compile 13m 18s branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚 compile 10m 45s branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚 checkstyle 2m 6s branch-2.10 passed
+1 💚 mvnsite 10m 46s branch-2.10 passed
-1 ❌ javadoc 0m 45s /branch-javadoc-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt root in branch-2.10 failed with JDK Azul Systems, Inc.-1.7.0_262-b10.
+1 💚 javadoc 5m 12s branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗 spotbugs 0m 39s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗 spotbugs 0m 18s branch/hadoop-assemblies no spotbugs output file (spotbugsXml.xml)
-1 ❌ spotbugs 22m 32s /branch-spotbugs-root-warnings.html root in branch-2.10 has 4 extant spotbugs warnings.
-1 ❌ spotbugs 1m 38s /branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings.
-1 ❌ spotbugs 2m 4s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html hadoop-hdfs-project/hadoop-hdfs in branch-2.10 has 1 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for patch
+1 💚 mvninstall 23m 4s the patch passed
+1 💚 compile 13m 19s the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚 javac 13m 19s the patch passed
+1 💚 compile 11m 0s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚 javac 11m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 2m 2s the patch passed
+1 💚 mvnsite 10m 54s the patch passed
+1 💚 xml 0m 31s The patch has no ill-formed XML file.
-1 ❌ javadoc 0m 26s /patch-javadoc-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt root in the patch failed with JDK Azul Systems, Inc.-1.7.0_262-b10.
+1 💚 javadoc 5m 17s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗 spotbugs 0m 16s hadoop-project has no data from spotbugs
+0 🆗 spotbugs 0m 17s hadoop-assemblies has no data from spotbugs
_ Other Tests _
-1 ❌ unit 432m 56s /patch-unit-root.txt root in the patch passed.
-1 ❌ asflicense 1m 20s /results-asflicense.txt The patch generated 2 ASF License warnings.
667m 10s
Reason Tests
Failed junit tests hadoop.io.compress.TestCompressorDecompressor
hadoop.util.TestClassUtil
hadoop.io.compress.snappy.TestSnappyCompressorDecompressor
hadoop.fs.sftp.TestSFTPFileSystem
hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints
hadoop.yarn.sls.appmaster.TestAMSimulator
hadoop.fs.azure.TestBlobMetadata
hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
hadoop.fs.azure.TestWasbFsck
hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
hadoop.fs.azure.TestOutOfBandAzureBlobOperations
hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
hadoop.fs.azure.TestNativeAzureFileSystemMocked
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/1/artifact/out/Dockerfile
GITHUB PR #4151
Optional Tests dupname asflicense codespell compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname Linux fb7c007a87de 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-2.10 / f241a09
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Multi-JDK versions /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/1/testReport/
Max. process+thread count 3935 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-assemblies hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-common-project/hadoop-nfs hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project hadoop-tools/hadoop-azure . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@iwasakims
Copy link
Member Author

The failure of TestClassUtil is relevant. I missed to include the change on backporting.

@iwasakims
Copy link
Member Author

Test failures related to wasb are reproducible even without the patch. I could not reproduce other failures on my local.

@iwasakims
Copy link
Member Author

I got no issue on manual testing on my local security-enabled-pseudo-distributed cluster including kms and httpfs.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 40s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ branch-2.10 Compile Tests _
+0 🆗 mvndep 3m 55s Maven dependency ordering for branch
+1 💚 mvninstall 14m 30s branch-2.10 passed
+1 💚 compile 13m 57s branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚 compile 11m 46s branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚 checkstyle 2m 15s branch-2.10 passed< 8000 /td>
+1 💚 mvnsite 12m 35s branch-2.10 passed
-1 ❌ javadoc 0m 47s /branch-javadoc-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt root in branch-2.10 failed with JDK Azul Systems, Inc.-1.7.0_262-b10.
+1 💚 javadoc 5m 49s branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗 spotbugs 0m 39s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗 spotbugs 0m 18s branch/hadoop-assemblies no spotbugs output file (spotbugsXml.xml)
-1 ❌ spotbugs 27m 28s /branch-spotbugs-root-warnings.html root in branch-2.10 has 4 extant spotbugs warnings.
-1 ❌ spotbugs 1m 58s /branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings.
-1 ❌ spotbugs 2m 20s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html hadoop-hdfs-project/hadoop-hdfs in branch-2.10 has 1 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for patch
+1 💚 mvninstall 21m 3s the patch passed
+1 💚 compile 13m 5s the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚 javac 13m 5s the patch passed
+1 💚 compile 10m 59s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚 javac 10m 59s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 2m 5s the patch passed
+1 💚 mvnsite 10m 54s the patch passed
+1 💚 xml 0m 32s The patch has no ill-formed XML file.
-1 ❌ javadoc 0m 26s /patch-javadoc-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt root in the patch failed with JDK Azul Systems, Inc.-1.7.0_262-b10.
+1 💚 javadoc 5m 24s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗 spotbugs 0m 14s hadoop-project has no data from spotbugs
+0 🆗 spotbugs 0m 14s hadoop-assemblies has no data from spotbugs
_ Other Tests _
-1 ❌ unit 435m 36s /patch-unit-root.txt root in the patch passed.
-1 ❌ asflicense 1m 20s /results-asflicense.txt The patch generated 2 ASF License warnings.
671m 20s
Reason Tests
Failed junit tests hadoop.io.compress.TestCompressorDecompressor
hadoop.io.compress.snappy.TestSnappyCompressorDecompressor
hadoop.fs.sftp.TestSFTPFileSystem
hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints
hadoop.fs.azure.TestBlobMetadata
hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
hadoop.fs.azure.TestWasbFsck
hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
hadoop.fs.azure.TestOutOfBandAzureBlobOperations
hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
hadoop.fs.azure.TestNativeAzureFileSystemMocked
hadoop.yarn.client.api.impl.TestAMRMProxy
hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/2/artifact/out/Dockerfile
GITHUB PR #4151
Optional Tests dupname asflicense codespell compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname Linux a370bd1143fa 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-2.10 / 4ead943
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Multi-JDK versions /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/2/testReport/
Max. process+thread count 3937 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-assemblies hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-common-project/hadoop-nfs hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project hadoop-tools/hadoop-azure . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4151/2/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@iwasakims
Copy link
Member Author

Warnings about javadoc, spotbugs and asflicense are not related (as expected since this is backport of #4084). I will merge this shortly.

@iwasakims iwasakims merged commit 6715746 into apache:branch-2.10 Apr 14, 2022
@arjun4084346 arjun4084346 mentioned this pull request Apr 28, 2022
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@iwasakims @hadoop-yetus
0