Make AST operators nullable #9096
Conversation
… to select the appropriate operator (fails for nullable operators now).
…o simplify dispatch.
…d test both EQUAL and NULL_EQUAL joins for different desired null equalities.
There was a problem hiding this comment.
Mostly minor suggestions for improvement. The design looks good overall.
One broader question: should we adopt a policy of only appending new AST operators to the end? Is there any possibility for users that rely on our C++ interface to see breakage from inserting new operators in the middle?
Inserting into the middle of an enum will not affect the API, but it is generally not ABI-safe, so yes this could break client code compiled against libcudf. However, at present libcudf does not even promise API compatibility between releases, so I don't think we need to worry about it. Expression evaluation is still very new and we made the last release with the expectation that we'd have to make many (potentially API- and/or ABI-breaking) changes in the subsequent release. We've made similarly ABI-breaking changes in other much more frequently used parts of the code base recently, so I think it's definitely OK to do in what's still a somewhat experimental feature in some ways. |
|
@gpucibot merge |
#9096 inadvertently removed the null equality parameter for the left semi and left anti equality joins in the JNI code. This restores the old behavior since #9096 only targeted conditional joins. Authors: - Jason Lowe (https://github.com/jlowe) Approvers: - Robert (Bobby) Evans (https://github.com/revans2) URL: #9207
Relates to #9851. This removes deprecated methods from the Java Table class, including APIs that would allow writing ORC files without specifying metadata via writer options needed for some columns (e.g.: precision for decimal columns). This also fixes a test where decimal precisions where not specified despite trying to write decimal columns to ORC. Javadoc errors introduced by #9096 were also corrected. Authors: - Jason Lowe (https://github.com/jlowe) Approvers: - Robert (Bobby) Evans (https://github.com/revans2) URL: #9853
Contributes to #8980 and #8145.
This PR makes all
operator_functorstemplated on the potential nullability of the inputs, allowing per-operator customization of null handling with minimal performance penalties. The default specialization in the nullable case is a null-propagating fall-through to the non-nullable case, which matches standard libcudf semantics, but any specific operator's null handling behavior can be customized by explicit specialization of theoperator_functortemplate corresponding to that operator.This PR implements
NULL_EQUAL,NULL_LOGICAL_AND, andNULL_LOGICAL_ORoperators that behave like their non-nullable counterparts when no nulls are present but can return non-null results from null inputs. TheNULL_EQUALoperator replaces thecompare_nullsparameter for conditional joins.