8000 HDDS-1216. Change name of ozoneManager service in docker compose file… by ajayydv · Pull Request #553 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HDDS-1216. Change name of ozoneManager service in docker compose file… #553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

ajayydv
Copy link
Contributor
@ajayydv ajayydv commented Mar 4, 2019

…s to om. Contributed by Ajay Kumar.

Secure S3 test Failure
Run Keyword Install aws cli
${rc} ${result} = Run And Return Rc And Output aws s3api --endpoint-url ${ENDPOINT_URL} create-bucket --bucket bucket-test123
Should Be True ${rc} > 0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whitespace:tabs in line

Execute kinit -k testuser/${hostname}@EXAMPLE.COM -t /etc/security/keytabs/testuser.keytab
${result} = Execute ozone sh s3 getsecret
${accessKey} = Get Regexp Matches ${result} (?<=awsAccessKey=).*
${secret} = Get Regexp Matches ${result} (?<=awsSecret=).*

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whitespace:tabs in line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this change from here? As this Jira purpose is not for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to move it to a separate Jira. Let's keep only ozoneManager -> om refactoring changes in this patch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed s3 related changes, HDDS-1043 will have them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still we have a few of them (commonawslib, docker-config)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elek thanks for catching that, removed from both files (commonawslib, docker-config).

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 20 Docker mode activated.
_ Prechecks _
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1091 trunk passed
-1 compile 24 dist in trunk failed.
-1 mvnsite 26 dist in trunk failed.
+1 shadedclient 706 branch has no errors when building and testing our client artifacts.
+1 javadoc 20 trunk passed
_ Patch Compile Tests _
-1 mvninstall 20 dist in the patch failed.
-1 compile 20 dist in the patch failed.
-1 javac 20 dist in the patch failed.
-1 mvnsite 21 dist in the patch failed.
+1 shellcheck 1 There were no new shellcheck issues.
+1 shelldocs 15 There were no new shelldocs issues.
-1 whitespace 0 The patch 3 line(s) with tabs.
+1 shadedclient 838 patch has no errors when building and testing our client artifacts.
+1 javadoc 17 the patch passed
_ Other Tests _
-1 unit 22 dist in the patch failed.
+1 asflicense 27 The patch does not generate ASF License warnings.
3001
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/Dockerfile
GITHUB PR #553
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient yamllint shellcheck shelldocs
uname Linux 1ea77581ba55 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / cb0fa0c
maven version: Apache Maven 3.3.9
Default Java 1.8.0_191
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/branch-compile-hadoop-ozone_dist.txt
mvnsite https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/branch-mvnsite-hadoop-ozone_dist.txt
shellcheck v0.4.6
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/patch-mvninstall-hadoop-ozone_dist.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/patch-compile-hadoop-ozone_dist.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/patch-compile-hadoop-ozone_dist.txt
mvnsite https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/patch-mvnsite-hadoop-ozone_dist.txt
whitespace https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/whitespace-tabs.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/artifact/out/patch-unit-hadoop-ozone_dist.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/testReport/
Max. process+thread count 340 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-553/1/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

Execute kinit -k testuser/${hostname}@EXAMPLE.COM -t /etc/security/keytabs/testuser.keytab
${result} = Execute ozone sh s3 getsecret
${accessKey} = Get Regexp Matches ${result} (?<=awsAccessKey=).*
${secret} = Get Regexp Matches ${result} (?<=awsSecret=).*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this change from here? As this Jira purpose is not for this.

Copy link
Contributor
@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And also can we fix jenkins checkstyle issues?

image: apache/hadoop-runner
privileged: true #required by the profiler
volumes:
- ../..:/opt/hadoop
ports:
- 9874:9874
environment:
ENSURE_OM_INITIALIZED: /data/metadata/ozoneManager/current/VERSION
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we have changed to om, will be this be same as service name when path is created?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed this for consistency purpose. IMO path can be independent of service name.

@@ -78,7 +78,7 @@ execute_tests(){
TITLE="Ozone $TEST tests with $COMPOSE_DIR cluster"
set +e
OUTPUT_NAME="$COMPOSE_DIR-${TEST//\//_}"
docker-compose -f "$COMPOSE_FILE" exec -T ozoneManager python -m robot --log NONE --report NONE "${OZONE_ROBOT_OPTS[@]}" --output "smoketest/$RESULT_DIR/robot-$OUTPUT_NAME.xml" --logtitle "$TITLE" --reporttitle "$TITLE" "smoketest/$TEST"
docker-compose -f "$COMPOSE_FILE" exec -T om python -m robot --log NONE --report NONE "${OZONE_ROBOT_OPTS[@]}" --output "smoketest/$RESULT_DIR/robot-$OUTPUT_NAME.xml" --logtitle "$TITLE" --reporttitle "$TITLE" "smoketest/$TEST"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whitespace:tabs in line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what is the problem here, change involves just rename.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, you modified a line which already contained a tab.

@hadoop-yetus

This comment has been minimized.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 25 Docker mode activated.
_ Prechecks _
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1098 trunk passed
+1 compile 71 trunk passed
+1 mvnsite 28 trunk passed
+1 shadedclient 644 branch has no errors when building and testing our client artifacts.
+1 javadoc 23 trunk passed
_ Patch Compile Tests _
-1 mvninstall 19 dist in the patch failed.
+1 compile 19 the patch passed
+1 javac 19 the patch passed
+1 mvnsite 22 the patch passed
+1 shellcheck 1 There were no new shellcheck issues.
+1 shelldocs 17 The patch generated 0 new + 104 unchanged - 136 fixed = 104 total (was 240)
-1 whitespace 4 The patch has 75 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-1 whitespace 5 The patch 19850 line(s) with tabs.
+1 shadedclient 993 patch has no errors when building and testing our client artifacts.
+1 javadoc 18 the patch passed
_ Other Tests _
+1 unit 24 dist in the patch passed.
+1 asflicense 31 The patch does not generate ASF License warnings.
3185
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/artifact/out/Dockerfile
GITHUB PR #553
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient yamllint shellcheck shelldocs
uname Linux da1508afe25b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 7f636b4
maven version: Apache Maven 3.3.9
Default Java 1.8.0_191
shellcheck v0.4.6
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/artifact/out/patch-mvninstall-hadoop-ozone_dist.txt
whitespace https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/artifact/out/whitespace-eol.txt
whitespace https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/artifact/out/whitespace-tabs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/testReport/
Max. process+thread count 414 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-553/3/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 26 Docker mode activated.
_ Prechecks _
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1124 trunk passed
+1 compile 87 trunk passed
+1 mvnsite 32 trunk passed
+1 shadedclient 614 branch has no errors when building and testing our client artifacts.
+1 javadoc 25 trunk passed
_ Patch Compile Tests _
-1 mvninstall 19 dist in the patch failed.
+1 compile 19 the patch passed
+1 javac 19 the patch passed
+1 mvnsite 20 the patch passed
+1 shellcheck 1 There were no new shellcheck issues.
+1 shelldocs 18 The patch generated 0 new + 104 unchanged - 136 fixed = 104 total (was 240)
-1 whitespace 3 The patch has 75 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-1 whitespace 4 The patch 19850 line(s) with tabs.
+1 shadedclient 894 patch has no errors when building and testing our client artifacts.
+1 javadoc 15 the patch passed
_ Other Tests _
+1 unit 22 dist in the patch passed.
+1 asflicense 28 The patch does not generate ASF License warnings.
3107
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/artifact/out/Dockerfile
GITHUB PR #553
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient yamllint shellcheck shelldocs
uname Linux 4db865cc78cf 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 1ad5bfc
maven version: Apache Maven 3.3.9
Default Java 1.8.0_191
shellcheck v0.4.6
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/artifact/out/patch-mvninstall-hadoop-ozone_dist.txt
whitespace https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/artifact/out/whitespace-eol.txt
whitespace https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/artifact/out/whitespace-tabs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/testReport/
Max. process+thread count 446 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-553/4/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Member
@elek elek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Tested locally, and all the acceptance tests are failed.

I can confirm that no new tabs are introduced (checked with git diff --word-diff=color --cached)

@elek elek closed this in 9d87247 Mar 6, 2019
@elek
Copy link
Member
elek commented Mar 6, 2019

Merged. Thanks you very much @ajayydv the contribution. I am very happy that we started to use the shorter names.

elek pushed a commit that referenced this pull request Mar 7, 2019
@ajayydv ajayydv deleted the HDDS-1216 branch March 21, 2019 22:29
shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
We require visibility into how effectively host-affinity performs. The goal is to help easily answer the following questions.
- How effectively is YARN matching my preferred-host requests
- When does Samza fallback to abandoning locality and issuing any-host requests?

design doc: https://docs.google.com/document/d/1oeNKDnG4JIGT2846us-jpnGW_RUjMjPIKDeEUlE_-jg/edit#

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Prateek M<pmaheshw@linkedin.com>

Closes apache#553 from vjagadish1989/hostaffinity-metrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0