HDDS-1232 : Recon Container DB service definition. #569

avijayanhwx · 2019-03-07T18:53:22Z

The attached patch contains the Recon container DB service definition (Interface + Impl). It uses a RocksDB instance to store the container-Key information. The interface is still evolving at this point of time, and maybe updated when the end to end work is done to read the OM DB snapshot and write the container-key data into this DB.

Unit testing done. Manual testing will be done along with HDDS-1233 and HDDS-1234.

hadoop-yetus · 2019-03-07T20:32:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	25	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	60	Maven dependency ordering for branch
+1	mvninstall	1087	trunk passed
+1	compile	967	trunk passed
+1	checkstyle	179	trunk passed
+1	mvnsite	116	trunk passed
+1	shadedclient	927	branch has no errors when building and testing our client artifacts.
+1	findbugs	133	trunk passed
+1	javadoc	78	trunk passed
		_ Patch Compile Tests _
0	mvndep	20	Maven dependency ordering for patch
+1	mvninstall	80	the patch passed
+1	compile	876	the patch passed
+1	javac	876	the patch passed
+1	checkstyle	184	the patch passed
+1	mvnsite	92	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	636	patch has no errors when building and testing our client artifacts.
-1	findbugs	42	hadoop-ozone/ozone-recon generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)
+1	javadoc	78	the patch passed
		_ Other Tests _
-1	unit	65	common in the patch failed.
+1	unit	33	framework in the patch passed.
+1	unit	24	ozone-recon in the patch passed.
+1	asflicense	33	The patch does not generate ASF License warnings.
		5802

Reason	Tests
FindBugs	module:hadoop-ozone/ozone-recon
	Found reliance on default encoding in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getCountForForContainerKeyPrefix(ContainerKeyPrefix):in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getCountForForContainerKeyPrefix(ContainerKeyPrefix): String.getBytes() At ContainerDBServiceProviderImpl.java:[line 79]
	Found reliance on default encoding in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long):in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long): new String(byte[]) At ContainerDBServiceProviderImpl.java:[line 102]
	Found reliance on default encoding in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long):in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long): String.getBytes() At ContainerDBServiceProviderImpl.java:[line 97]
	Found reliance on default encoding in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.storeContainerKeyMapping(ContainerKeyPrefix, Integer):in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.storeContainerKeyMapping(ContainerKeyPrefix, Integer): String.getBytes() At ContainerDBServiceProviderImpl.java:[line 61]
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-569/1/artifact/out/Dockerfile
GITHUB PR	#569
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux d0fa55847bd2 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `1bc282e`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/1/artifact/out/new-findbugs-hadoop-ozone_ozone-recon.html
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/1/artifact/out/patch-unit-hadoop-hdds_common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/1/testReport/
Max. process+thread count	445 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/framework hadoop-ozone/ozone-recon U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/1/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

…es).

hadoop-yetus · 2019-03-07T22:47:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	26	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	25	Maven dependency ordering for branch
+1	mvninstall	1025	trunk passed
+1	compile	973	trunk passed
+1	checkstyle	193	trunk passed
+1	mvnsite	133	trunk passed
+1	shadedclient	1046	branch has no errors when building and testing our client artifacts.
+1	findbugs	169	trunk passed
+1	javadoc	119	trunk passed
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	96	the patch passed
+1	compile	941	the patch passed
+1	javac	941	the patch passed
+1	checkstyle	187	the patch passed
+1	mvnsite	103	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	612	patch has no errors when building and testing our client artifacts.
-1	findbugs	45	hadoop-ozone/ozone-recon generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	javadoc	90	the patch passed
		_ Other Tests _
-1	unit	68	common in the patch failed.
+1	unit	38	framework in the patch passed.
+1	unit	34	ozone-recon in the patch passed.
+1	asflicense	35	The patch does not generate ASF License warnings.
		6054

Reason	Tests
FindBugs	module:hadoop-ozone/ozone-recon
	Found reliance on default encoding in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long):in org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl.getKeyPrefixesForContainer(long): new String(byte[]) At ContainerDBServiceProviderImpl.java:[line 120]
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-569/2/artifact/out/Dockerfile
GITHUB PR	#569
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 773b2c016654 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `1bc282e`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/2/artifact/out/new-findbugs-hadoop-ozone_ozone-recon.html
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/2/artifact/out/patch-unit-hadoop-hdds_common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/2/testReport/
Max. process+thread count	412 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/framework hadoop-ozone/ozone-recon U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/2/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

…ation issue).

hadoop-yetus · 2019-03-08T00:39:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	28	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	23	Maven dependency ordering for branch
+1	mvninstall	1016	trunk passed
+1	compile	942	trunk passed
+1	checkstyle	197	trunk passed
+1	mvnsite	130	trunk passed
+1	shadedclient	1056	branch has no errors when building and testing our client artifacts.
+1	findbugs	175	trunk passed
+1	javadoc	115	trunk passed
		_ Patch Compile Tests _
0	mvndep	21	Maven dependency ordering for patch
+1	mvninstall	92	the patch passed
+1	compile	909	the patch passed
+1	javac	909	the patch passed
+1	checkstyle	199	the patch passed
+1	mvnsite	112	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	608	patch has no errors when building and testing our client artifacts.
+1	findbugs	175	the patch passed
+1	javadoc	85	the patch passed
		_ Other Tests _
-1	unit	67	common in the patch failed.
+1	unit	39	framework in the patch passed.
-1	unit	25	ozone-recon in the patch failed.
+1	asflicense	41	The patch does not generate ASF License warnings.
		5990

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-569/3/artifact/out/Dockerfile
GITHUB PR	#569
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 29efa507691f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `064f38b`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/3/artifact/out/patch-unit-hadoop-hdds_common.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/3/artifact/out/patch-unit-hadoop-ozone_ozone-recon.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/3/testReport/
Max. process+thread count	444 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/framework hadoop-ozone/ozone-recon U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/3/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-03-08T00:50:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	38	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	80	Maven dependency ordering for branch
+1	mvninstall	1128	trunk passed
+1	compile	1101	trunk passed
+1	checkstyle	202	trunk passed
+1	mvnsite	140	trunk passed
+1	shadedclient	1089	branch has no errors when building and testing our client artifacts.
+1	findbugs	168	trunk passed
+1	javadoc	105	trunk passed
		_ Patch Compile Tests _
0	mvndep	60	Maven dependency ordering for patch
+1	mvninstall	91	the patch passed
+1	compile	933	the patch passed
+1	javac	933	the patch passed
+1	checkstyle	198	the patch passed
+1	mvnsite	116	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	2	The patch has no ill-formed XML file.
+1	shadedclient	691	patch has no errors when building and testing our client artifacts.
+1	findbugs	184	the patch passed
+1	javadoc	105	the patch passed
		_ Other Tests _
-1	unit	91	common in the patch failed.
+1	unit	42	framework in the patch passed.
+1	unit	42	ozone-recon in the patch passed.
+1	asflicense	52	The patch does not generate ASF License warnings.
		6600

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-569/4/artifact/out/Dockerfile
GITHUB PR	#569
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 59fde664740b 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `064f38b`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/4/artifact/out/patch-unit-hadoop-hdds_common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/4/testReport/
Max. process+thread count	340 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/framework hadoop-ozone/ozone-recon U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/4/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

…er test run).

hadoop-yetus · 2019-03-08T02:51:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	27	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	19	Maven dependency ordering for branch
-1	mvninstall	20	root in trunk failed.
-1	compile	21	root in trunk failed.
-0	checkstyle	18	The patch fails to run checkstyle in root
-1	mvnsite	21	common in trunk failed.
-1	mvnsite	20	framework in trunk failed.
-1	mvnsite	20	ozone-recon in trunk failed.
+1	shadedclient	100	branch has no errors when building and testing our client artifacts.
-1	findbugs	21	common in trunk failed.
+1	javadoc	58	trunk passed
		_ Patch Compile Tests _
0	mvndep	59	Maven dependency ordering for patch
+1	mvninstall	84	the patch passed
-1	compile	575	root in the patch failed.
-1	javac	575	root in the patch failed.
+1	checkstyle	194	the patch passed
+1	mvnsite	85	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
-1	shadedclient	657	patch has errors when building and testing our client artifacts.
+1	findbugs	160	the patch passed
+1	javadoc	70	the patch passed
		_ Other Tests _
-1	unit	76	common in the patch failed.
+1	unit	31	framework in the patch passed.
+1	unit	23	ozone-recon in the patch passed.
+1	asflicense	31	The patch does not generate ASF License warnings.
		2599

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/Dockerfile
GITHUB PR	#569
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux cf85033f1891 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `373705f`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-mvninstall-root.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-compile-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out//testptch/patchprocess/maven-branch-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-mvnsite-hadoop-hdds_common.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-mvnsite-hadoop-hdds_framework.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-mvnsite-hadoop-ozone_ozone-recon.txt
findbugs	v3.1.0-RC1
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/branch-findbugs-hadoop-hdds_common.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/patch-compile-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/patch-compile-root.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/artifact/out/patch-unit-hadoop-hdds_common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/testReport/
Max. process+thread count	411 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/framework hadoop-ozone/ozone-recon U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-569/5/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

HDDS-1232 : Recon Container DB service definition.

e818e55

bharatviswa504 added the ozone label Mar 7, 2019

swagle approved these changes Mar 7, 2019

View reviewed changes

HDDS-1232 : Recon Container DB service definition. (Fix findbugs issu…

2edafe1

…es).

Aravindan Vijayan added 2 commits March 7, 2019 14:56

HDDS-1232 Recon Container DB service definition. (Fix Internationaliz…

e1587a3

…ation issue).

HDDS-1232 Recon Container DB service definition (Remove unused import).

306a18a

HDDS-1232 Recon Container DB service definition (Test commit to trigg…

06c37a7

…er test run).

avijayanhwx closed this Mar 8, 2019

avijayanhwx deleted the HDDS-1232-trunk branch March 8, 2019 21:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-1232 : Recon Container DB service definition. #569

HDDS-1232 : Recon Container DB service definition. #569

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HDDS-1232 : Recon Container DB service definition. #569

HDDS-1232 : Recon Container DB service definition. #569

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!