8000 Coral-Hive: Roll back SqlToRelConverter to be a singleton field in HiveToRelConverter by ljfgem · Pull Request #231 · linkedin/coral · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Coral-Hive: Roll back SqlToRelConverter to be a singleton field in HiveToRelConverter #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 14, 2022
Merged

Coral-Hive: Roll back SqlToRelConverter to be a singleton field in HiveToRelConverter #231

merged 1 commit into from
Feb 14, 2022

Conversation

ljfgem
Copy link
Collaborator
@ljfgem ljfgem commented Feb 14, 2022

#137 changed SqlToRelConverter to be a non-singleton field in d84241c#diff-68fe9452b841d5fb41f60f2b66aee34c41aeb7eb9681fd4f0d40a85c9dc5197fR246, which caused tricky issue while translating some internal views.
i.e.

Exception in thread "main" java.lang.RuntimeException: while converting `a`.`b` = `c`.`d`.`e`
        at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$0(ReflectiveConvertletTable.java:86)
        at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:63)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4790)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4092)
        at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4659)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertJoinCondition(SqlToRelConverter.java:2708)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2100)
        at com.linkedin.coral.hive.hive2rel.HiveSqlToRelConverter.convertFrom(HiveSqlToRelConverter.java:86)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:646)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:627)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3181)
        at com.linkedin.coral.hive.hive2rel.HiveSqlToRelConverter.convertQuery(HiveSqlToRelConverter.java:63)
        at com.linkedin.coral.common.ToRelConverter.toRel(ToRelConverter.java:166)
        at com.linkedin.coral.common.ToRelConverter.convertView(ToRelConverter.java:133)
        at com.linkedin.coral.tools.TrinoValidator.convertAndValidate(TrinoValidator.java:61)
        at com.linkedin.coral.tools.SingleViewTranslation.translateTable(SingleViewTranslation.java:63)
        at com.linkedin.coral.tools.SingleViewTranslation.main(SingleViewTranslation.java:55)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$0(ReflectiveConvertletTable.java:83)
        ... 17 more
Caused by: java.lang.NullPointerException
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3721)
        at org.apache.calcite.sql2rel.SqlToRelConverter.access$2300(SqlToRelConverter.java:217)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4799)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4092)
        at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:317)
        at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4659)
        at org.apache.calcite.sql2rel.StandardConvertletTable.convertExpressionList(StandardConvertletTable.java:793)
        at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:769)
        at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:756)
        ... 22 more

This PR is to roll this change back.

Tests:

  1. Modified unit tests
  2. Tested on the affected views, could be translated well with this PR
  3. Integration tests for Trino (enhanced), Spark and Avro, no regression in any tests

funcheetah
funcheetah approved these changes Feb 14, 2022
View reviewed changes
Copy link
Contributor
@funcheetah funcheetah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ljfgem for the quick fix. LGTM.

@ljfgem ljfgem merged commit 13a8c40 into linkedin:master Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@funcheetah funcheetah funcheetah approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
43B4
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ljfgem @funcheetah
0