HADOOP-19329. Remove usage of sun.misc.Signal #7145
+76
−43
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
jojochuang
reviewed
Nov 6, 2024
There was a problem hiding this comment.
Would you like to open a new jira? HADOOP-19318 is resolved as workaround.
| @@ -416,6 +416,12 @@ | |||
| <artifactId>lz4-java</artifactId> | |||
| <scope>provided</scope> | |||
| </dependency> | |||
| <dependency> | |||
| <groupId>com.github.jnr</groupId> | |||
There was a problem hiding this comment.
the license is tri EPL/GPL/LGPL license. Should be fine.
There was a problem hiding this comment.
Thanks for your review, and i have added com.github.jnr to GPLv2 module in LICENSE-binary file.
There was a problem hiding this comment.
ASF should use this under the Eclipse Public License (Category B). GPL and LGPL are incompatible with the Apache license (Category X).
|
shadedclient is broken. That needs to be fixed. I am getting this error compiling with JDK8: [ERROR] Rule 0: org.apache.maven.enforcer.rules.dependency.DependencyConvergence failed with message: |
OK. My Jira account will be granted permissions soon, and I’ll create a new Jira right after that. |
I have excluded the conflicting dependencies related to the jnr-posix module,could you please help review this change again?Many thanks. |
|
@slfan1989 Could you please help review this PR? Thank you very much. |
jojochuang
reviewed
Nov 6, 2024
LICENSE-binary
Outdated
| @@ -498,6 +498,7 @@ org.slf4j:slf4j-reload4j:1.7.36 | |||
| CDDL 1.1 + GPLv2 with classpath exception | |||
| ----------------------------------------- | |||
|
|
|||
| com.github.jnr:jnr-posix:3.1.19 | |||
There was a problem hiding this comment.
actually it's not CDDL 1.1 + GPLv2 with classpath exception. Eclipse Public License 1/.02.0 is acceptable though. I think let's add a new section for "Eclipse Public License 2.0" under "Eclipse Public License 1.0"
For more details, check out Apache's thirdparty license policy: https://www.apache.org/legal/resolved.html
There was a problem hiding this comment.
Done.You are right, I added EPL 2.0 section.
|
💔 -1 overall
This message was automatically generated. |
Opened a new JIRA HADOOP-19329. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
pan3793
reviewed
Nov 8, 2024
LICENSE-binary
Outdated
| Eclipse Public License 2.0 | ||
| -------------------------- | ||
|
|
||
| com.github.jnr:jnr-posix:3.1.19 |
There was a problem hiding this comment.
it also pulls transitive jars.
Honestly, it's a bit heavy to pull bunches of jars to just work around the access of sun.misc.Signal API. Could it be addressed by using reflection?
There was a problem hiding this comment.
@pan3793 This change is likely based on related modifications in Ozone. If reflection is used, could you provide some specific implementation ideas?
Ozone:
JIRA: HDDS-11078. Remove usage of sun.misc.Signal
apache/ozone#7006
apache/ozone#6876
@myandpr Is there any additional information?
There was a problem hiding this comment.
Exactly, and the problem is that sun.misc package is no longer supported after JDK 9，even removed in a future release( related information: https://openjdk.org/jeps/260). @slfan1989 @pan3793
|
@jojochuang Could you help review this PR again? Thank you very much! @myandpr Can we trigger the compilation again? |
3350605 to
f8b392d
Compare
fine, I have triggered. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@adoroszlai Could you please help review this PR? Thank you very much! |
adoroszlai
reviewed
Dec 9, 2024
There was a problem hiding this comment.
Thanks @myandpr for the patch. The change in SignalLogger looks good. I cannot comment on the rest (exlusions and IrqHandler).
hadoop-project/pom.xml
Outdated
| @@ -237,6 +237,7 @@ | |||
| <yarnpkg.version>v1.22.5</yarnpkg.version> | |||
| <apache-ant.version>1.10.13</apache-ant.version> | |||
| <jmh.version>1.20</jmh.version> | |||
| <jnr.posix.version>3.1.19</jnr.posix.version> | |||
There was a problem hiding this comment.
3.1.20 is now available.
| log.error("RECEIVED SIGNAL " + signal.getNumber() + | ||
| ": SIG" + signal.getName()); | ||
| public void handle(int signal) { | ||
| log.error("RECEIVED SIGNAL {}: SIG{}", signal, Signal.valueOf(signal)); |
There was a problem hiding this comment.
Signal name includes SIG..., so I think this would log e.g. SIGSIGHUP.
Suggested change
| log.error("RECEIVED SIGNAL {}: SIG{}", signal, Signal.valueOf(signal)); | |
| log.error("RECEIVED SIGNAL {}: {}", signal, Signal.valueOf(signal)); |
There was a problem hiding this comment.
Thanks, you are right, I have fixed it.
Thanks for your review! |
530ced9 to
1b1dde1
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@myandpr Can we rebase this PR so that we can continue following up on it? |
steveloughran
requested changes
Jan 24, 2025
| @@ -159,6 +159,7 @@ | |||
| <exclude>org.xerial.snappy:*</exclude> | |||
| <!-- leave out kotlin classes --> | |||
| <exclude>org.jetbrains.kotlin:*</exclude> | |||
| <exclude>com.github.jnr:*</exclude> | |||
There was a problem hiding this comment.
what breaks once this is excluded? the service stuff should be ok as it the irq stuff is only used server side AFAIK.
| @@ -93,7 +94,7 @@ public void testBlockingShutdownTimeouts() throws Throwable { | |||
| InterruptEscalator escalator = new InterruptEscalator(launcher, 500); | |||
| // call the interrupt operation directly | |||
| try { | |||
| escalator.interrupted(new IrqHandler.InterruptData("INT", 3)); | |||
| escalator.interrupted(new IrqHandler.InterruptData("SIGINT", 3)); | |||
There was a problem hiding this comment.
we should move to LambdaTestUtils.intercept() here instead of the try/catch clause
assertExceptionDetails(EXIT_INTERRUPTED, "",
intercept(ExitUtil.ExitException.class, "", () ->
escalator.interrupted(new IrqHandler.InterruptData("SIGINT", 3))));
4 tasks
|
Hi, I managed to use reflection to replace the direct call of I think the only advantage is making us closer to compile Hadoop using higher JDK (e.g. 17) with cc @adoroszlai @slfan1989 @steveloughran @jojochuang @cnauroth |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
JIRA: HADOOP-19329. Remove usage of sun.misc.Signal
Fix compilation errors during the upgrade process, because sun.misc is not supported after jdk11.
we replace sun.misc.Signa 8000 l and sum.misc.SignalHandler using jnr-posix
How was this patch tested?
exist ut test
For code changes:
replace sun.misc.Signal and sum.misc.SignalHandler using jnr.constants.platform.Signal and jnr.posix.SignalHandler in jnr-posix