HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. #6517

hfutatzhanghb · 2024-02-01T03:48:14Z

Description of PR

Currently, when we write ec files. It can tolerate with at most numAllBlocks - numDataBlocks failed streamers.
However, this can lead to block missing exception in some case.

For example, When we use RS-6-3-1024K ec policy, we have written 6 blocks successfully.
this block is going to be reconstructed, but before that, one of the 6 blocks lost, we may lose the data forerver.
Or if we restart datanode at that time, client will get block missing exception.

So, we should better add some redundancy here.

hfutatzhanghb · 2024-02-01T03:57:17Z

@Hexiaoqiao @zhangshuyan0 @tasanuma @tomscut Hi, sir. Could you please help me review this PR when you are free ? Thanks a lot!

hadoop-yetus · 2024-02-01T08:58:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 18s		trunk passed
+1 💚	compile	2m 57s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	2m 47s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 41s		trunk passed
+1 💚	mvnsite	1m 9s		trunk passed
+1 💚	javadoc	1m 0s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 31s		trunk passed
+1 💚	shadedclient	22m 53s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 0s		the patch passed
+1 💚	compile	2m 48s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	2m 48s		the patch passed
+1 💚	compile	2m 50s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	2m 50s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3)
+1 💚	mvnsite	1m 4s		the patch passed
+1 💚	javadoc	0m 51s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 30s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 44s		the patch passed
+1 💚	shadedclient	23m 4s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 41s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	196m 45s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 24s		The patch does not generate ASF License warnings.
		308m 58s

Reason	Tests
Failed junit tests	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/1/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 5224dfd520de 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `96bf0b1`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubu 8000 ntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/1/testReport/
Max. process+thread count	4029 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-02-01T09:16:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 23s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 40s		trunk passed
+1 💚	compile	3m 11s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	3m 3s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	1m 15s		trunk passed
+1 💚	javadoc	0m 59s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	2m 58s		trunk passed
+1 💚	shadedclient	23m 0s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 19s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 12s		the patch passed
+1 💚	compile	3m 14s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	3m 14s		the patch passed
+1 💚	compile	2m 58s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	2m 58s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 36s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3)
+1 💚	mvnsite	1m 1s		the patch passed
+1 💚	javadoc	0m 47s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 9s		the patch passed
+1 💚	shadedclient	23m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 54s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	204m 46s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 28s		The patch does not generate ASF License warnings.
		319m 14s

Reason	Tests
Failed junit tests	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.namenode.TestReconstructStripedBlocks

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/2/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 30ad486a435c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `510fce0`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/2/testReport/
Max. process+thread count	3950 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-02-01T17:21:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 18s		trunk passed
+1 💚	compile	2m 52s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	2m 44s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	1m 16s		trunk passed
+1 💚	javadoc	1m 3s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 5s		trunk passed
+1 💚	shadedclient	20m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 3s		the patch passed
+1 💚	compile	2m 49s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	2m 49s		the patch passed
+1 💚	compile	2m 45s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	2m 45s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	0m 33s		the patch passed
+1 💚	mvnsite	1m 0s		the patch passed
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	2m 59s		the patch passed
+1 💚	shadedclient	20m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 53s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	200m 51s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		305m 34s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/3/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 2cb7ceaf1cb2 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `60c23a4`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/3/testReport/
Max. process+thread count	3937 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

zhangshuyan0

This is a good idea for me. I think we should allow different EC strategies to use different sizes of extraStreamerRedunency. May be a structure like ReplaceDatanodeOnFailure? Or may just contain EC schema name in configuration key?

hfutatzhanghb · 2024-02-07T09:42:49Z

This is a good idea for me. I think we should allow different EC strategies to use different sizes of extraStreamerRedunency. May be a structure like ReplaceDatanodeOnFailure? Or may just contain EC schema name in configuration key?

Nice opinion, Sir. Give me some time to think about how to design it. Hope we can push this idea forward. Thanks a lot!

hadoop-yetus · 2024-02-23T13:05:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 58s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 45s		trunk passed
+1 💚	compile	2m 57s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	2m 50s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 46s		trunk passed
+1 💚	mvnsite	1m 18s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 35s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌	spotbugs	1m 28s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	20m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 2s		the patch passed
+1 💚	compile	2m 51s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	2m 51s		the patch passed
+1 💚	compile	2m 43s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	2m 43s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 28 new + 33 unchanged - 0 fixed = 61 total (was 33)
+1 💚	mvnsite	1m 7s		the patch passed
+1 💚	javadoc	0m 55s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 25s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 18s		the patch passed
+1 💚	shadedclient	23m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 49s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	84m 49s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+0 🆗	asflicense	0m 24s		ASF License check generated no output?
		201m 0s

Reason	Tests
Failed junit tests	hadoop.hdfs.shortcircuit.TestShortCircuitCache
	hadoop.hdfs.server.blockmanagement.TestReplicationPolicyExcludeSlowNodes
	hadoop.hdfs.server.mover.TestStorageMover
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.TestDatanodeLayoutUpgrade
	hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
	hadoop.hdfs.tools.TestDebugAdmin
	hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
	hadoop.hdfs.TestErasureCodeBenchmarkThroughput
	hadoop.hdfs.TestDatanodeDeath
	hadoop.hdfs.server.mover.TestMover

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/4/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux cb17842e53c6 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `55fcce0`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/4/testReport/
Max. process+thread count	3083 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

tasanuma · 2024-02-26T15:56:28Z

hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

@@ -3908,6 +3908,18 @@
  </description>
 </property>

+<property>
+  <name>dfs.client.ec.EXAMPLEECPOLICYNAME.checkstreamer.redunency</name>


@hfutatzhanghb Thanks for the PR. I think it's a good feature.

In my honest opinion, dfs.client.ec.EXAMPLEECPOLICYNAME.checkstreamer.redunency is counter-intuitive. I prefer a setting that interprets values in a reverse way. In other words, it would be something like dfs.client.ec.EXAMPLEECPOLICYNAME.failed.write.block.tolerated, where if the value is 0, then no failures are tolerated. And if it's 3, we can tolerate up to 3 failures in block writing. If the setting value is empty (it would be default), we can tolerate failures up to the number of parity blocks. This is just my personal view.

@tasanuma Sir, thanks a lot for your reviewing and valuable opinion. Agree with you, We should make the configuration become more intuitive.

@tasanuma @zhangshuyan0 Hi, sir. Sorry for response too late here. I have uploaded a new patch according to the review opinions, please help me review this PR when you have bandwidth. Thanks a lot~

hadoop-yetus · 2024-03-21T21:18:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 8s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 51s		trunk passed
+1 💚	compile	3m 37s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	2m 59s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 47s		trunk passed
+1 💚	mvnsite	1m 19s		trunk passed
+1 💚	javadoc	1m 4s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 39s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	1m 35s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	22m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 20s		the patch passed
+1 💚	compile	3m 35s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	3m 35s		the patch passed
+1 💚	compile	3m 2s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	3m 2s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 35s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 29s		the patch passed
+1 💚	shadedclient	21m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 49s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	224m 46s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 28s		The patch does not generate ASF License warnings.
		338m 55s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedInputStream
	hadoop.hdfs.TestReadStripedFileWithDNFailure
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.TestErasureCodingMultipleRacks
	hadoop.hdfs.TestDecommissionWithStriped
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.hdfs.TestInjectionForSimulatedStorage
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/6/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 1e170073923f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d4701f0`
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/6/testReport/
Max. process+thread count	4222 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-03-21T21:25:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 6s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 3s		trunk passed
+1 💚	compile	3m 1s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	3m 25s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 51s		trunk passed
+1 💚	mvnsite	1m 34s		trunk passed
+1 💚	javadoc	1m 7s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 39s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	1m 33s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	22m 10s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 8s		the patch passed
+1 💚	compile	3m 3s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	3m 3s		the patch passed
+1 💚	compile	3m 27s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	3m 27s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 43s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 55s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 31s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 31s		the patch passed
+1 💚	shadedclient	22m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 45s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	233m 47s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 32s		The patch does not generate ASF License warnings.
		355m 42s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.TestReadStripedFileWithDNFailure
	hadoop.hdfs.TestErasureCodingPolicies
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy
	hadoop.hdfs.TestReadStripedFileWithDecoding
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.hdfs.TestMaintenanceState
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.namenode.TestCacheDirectives

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/5/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 9d3b481cd531 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d4701f0`
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/5/testReport/
Max. process+thread count	4766 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-03-22T08:35:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 7s		trunk passed
+1 💚	compile	3m 42s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	3m 3s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 47s		trunk passed
+1 💚	mvnsite	1m 20s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 35s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	1m 34s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	21m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 21s		the patch passed
+1 💚	compile	3m 31s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	3m 31s		the patch passed
+1 💚	compile	3m 1s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	3m 1s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 41s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚	mvnsite	1m 9s		the patch passed
+1 💚	javadoc	0m 55s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 33s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 26s		the patch passed
+1 💚	shadedclient	22m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 51s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	202m 48s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		317m 25s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedInputStream
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/8/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 7a0a80935d1e 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aca8b62`
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/8/testReport/
Max. process+thread count	3969 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-03-22T08:40:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 52s		trunk passed
+1 💚	compile	2m 55s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	3m 38s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 47s		trunk passed
+1 💚	mvnsite	1m 21s		trunk passed
+1 💚	javadoc	1m 12s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 38s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	1m 32s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	22m 27s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 5s		the patch passed
+1 💚	compile	3m 7s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	3m 7s		the patch passed
+1 💚	compile	3m 25s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	3m 25s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 41s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 28s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 32s		the patch passed
+1 💚	shadedclient	21m 59s		patch has no error A3E2 s when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 47s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	210m 9s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		324m 29s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.TestQuotaByStorageType
	hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
	hadoop.hdfs.TestReconstructStripedFileWithValidator
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/7/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux eb47352fabf4 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aca8b62`
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/7/testReport/
Max. process+thread count	3677 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-03-25T11:21:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 33s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 13s		trunk passed
+1 💚	compile	2m 57s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	2m 50s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	1m 16s		trunk passed
+1 💚	javadoc	1m 10s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 40s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	1m 28s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	20m 53s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 6s		the patch passed
+1 💚	compile	2m 49s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	2m 49s		the patch passed
+1 💚	compile	2m 52s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	2m 52s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 34 unchanged - 0 fixed = 35 total (was 34)
+1 💚	mvnsite	1m 3s		the patch passed
+1 💚	javadoc	0m 46s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 14s		the patch passed
+1 💚	shadedclient	20m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 48s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	203m 53s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		311m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/9/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 9673203f7816 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2a2bcd2`
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/9/testReport/
Max. process+thread count	4349 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/9/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hfutatzhanghb · 2024-03-25T12:39:59Z

The checkstyle warning can be ignored.

hadoop-yetus

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
			_ Prechecks _
+1 💚	dupname	0m 01s		No case conflicting files found.
+0 🆗	spotbugs	0m 01s		spotbugs executables are not available.
+0 🆗	codespell	0m 01s		codespell was not available.
+0 🆗	detsecrets	0m 01s		detect-secrets was not available.
+0 🆗	xmllint	0m 01s		xmllint was not available.
+1 💚	@author	0m 00s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 00s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	3m 29s		Maven dependency ordering for branch
+1 💚	mvninstall	128m 55s		trunk passed
+1 💚	compile	34m 05s		trunk passed
+1 💚	checkstyle	8m 09s		trunk passed
+1 💚	mvnsite	18m 54s		trunk passed
+1 💚	javadoc	17m 01s		trunk passed
+1 💚	shadedclient	230m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	3m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	12m 14s		the patch passed
+1 💚	compile	30m 00s		the patch passed
+1 💚	javac	30m 00s		the patch passed
-1 ❌	blanks	0m 00s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	4m 39s		the patch passed
+1 💚	mvnsite	12m 31s		the patch passed
+1 💚	javadoc	10m 33s		the patch passed
+1 💚	shadedclient	227m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	8m 34s		The patch does not generate ASF License warnings.
		689m 38s

Subsystem	Report/Notes
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	MINGW64_NT-10.0-17763 83306cec9003 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys
Build tool	maven
Personality	/c/hadoop/dev-support/bin/hadoop.sh
git revision	trunk / `2a2bcd2`
Default Java	Azul Systems, Inc.-1.8.0_332-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6517/1/testReport/
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6517/1/console
versions	git=2.44.0.windows.1
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…lures to prevent data loss.

hfutatzhanghb · 2024-12-31T15:26:38Z

@Hexiaoqiao @zhangshuyan0 @tasanuma Sir, sorry for pushing this PR forward too late. Could you please help review this PR when you are free? Thanks.

hadoop-yetus · 2024-12-31T20:08:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 19s		Maven dependency ordering for branch
+1 💚	mvninstall	17m 47s		trunk passed
+1 💚	compile	2m 47s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	compile	2m 43s		trunk passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	1m 15s		trunk passed
+1 💚	javadoc	1m 11s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 34s		trunk passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	spotbugs	2m 58s		trunk passed
+1 💚	shadedclient	20m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 0s		the patch passed
+1 💚	compile	2m 45s		the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javac	2m 45s		the patch passed
+1 💚	compile	2m 39s		the patch passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	javac	2m 39s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 33s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 34 unchanged - 0 fixed = 35 total (was 34)
+1 💚	mvnsite	1m 6s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	spotbugs	3m 0s		the patch passed
+1 💚	shadedclient	19m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 55s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	197m 8s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		290m 36s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.TestDecommission

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/10/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 4e7c21a445f9 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4daec24`
Default Java	Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/10/testReport/
Max. process+thread count	3701 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/10/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-01-01T08:18:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	17m 55s		trunk passed
+1 💚	compile	2m 45s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	compile	2m 43s		trunk passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	1m 14s		trunk passed
+1 💚	javadoc	1m 10s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	spotbugs	2m 53s		trunk passed
+1 💚	shadedclient	19m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 2s		the patch passed
+1 💚	compile	2m 45s		the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javac	2m 45s		the patch passed
+1 💚	compile	2m 34s		the patch passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	javac	2m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 35s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 34 unchanged - 0 fixed = 35 total (was 34)
+1 💚	mvnsite	1m 7s		the patch passed
+1 💚	javadoc	0m 56s		the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 27s		the patch passed with JDK Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
+1 💚	spotbugs	2m 53s		the patch passed
+1 💚	shadedclient	19m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 51s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	198m 1s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		291m 36s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDecommissionWithBackoffMonitor

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/11/artifact/out/Dockerfile
GITHUB PR	#6517
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 9dce009f121b 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ec25a05`
Default Java	Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~~us1-0ubuntu2~~20.04-ga
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/11/testReport/
Max. process+thread count	3806 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6517/11/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hfutatzhanghb · 2025-01-01T09:24:51Z

The checkstyle warning can be ignored.

github-actions bot added HDFS trunk labels Feb 1, 2024

hfutatzhanghb force-pushed the HDFS-17365 branch from 96bf0b1 to 510fce0 Compare February 1, 2024 03:56

hfutatzhanghb force-pushed the HDFS-17365 branch from 510fce0 to 60c23a4 Compare February 1, 2024 12:14

zhangshuyan0 reviewed Feb 7, 2024

View reviewed changes

hfutatzhanghb force-pushed the HDFS-17365 branch from 60c23a4 to 55fcce0 Compare February 23, 2024 09:43

tasanuma reviewed Feb 26, 2024

View reviewed changes

hfutatzhanghb force-pushed the HDFS-17365 branch 2 times, most recently from d73f2ca to d4701f0 Compare March 21, 2024 15:38

hfutatzhanghb force-pushed the HDFS-17365 branch from 68e46d9 to aca8b62 Compare March 22, 2024 03:17

hfutatzhanghb added 3 commits December 31, 2024 23:16

HDFS-17365. EC: Add extra redunency configuration in checkStreamerFai…

fae585a

…lures to prevent data loss.

trigger yetus.

4037fbe

fix failed unit tests.

4daec24

hfutatzhanghb force-pushed the HDFS-17365 branch from 2a2bcd2 to 4daec24 Compare December 31, 2024 15:16

hfutatzhanghb requested review from zhangshuyan0 and tasanuma December 31, 2024 15:22

fix blank line

ec25a05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. #6517

HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. #6517

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. #6517

Are you sure you want to change the base?

HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. #6517

Conversation

Description of PR

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!