Null-handling for Transforms #18845
Merged
Null-handling for Transforms #18845
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
davidwendt
reviewed
May 16, 2025
|
/ok to test d8b0433 |
|
/ok to test 8dded0a |
|
/ok to test 08ecef2 |
davidwendt
reviewed
May 28, 2025
davidwendt
reviewed
May 29, 2025
davidwendt
approved these changes
May 29, 2025
kingcrimsontianyu
approved these changes
May 29, 2025
mhaseeb123
approved these changes
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
ttnghia
reviewed
May 29, 2025
Co-authored-by: Nghia Truong <7416935+ttnghia@users.noreply.github.com>
ttnghia
approved these changes
May 29, 2025
|
/merge |
TomAugspurger
pushed a commit
to TomAugspurger/pygdf
that referenced
this pull request
May 30, 2025
This merge request implements null-handling for Transforms. The implementation outputs a null row if any of its input rows are null. The output bitmask is computed in a single pass using the null-masks of the input columns before the transform function is run on the valid output rows. It also handles the scalar input edge cases: - Scalar is not null; implies it is valid across the base column's size. We also can't use its bitmask as it is a bitmask of size 1, otherwise, we'd encounter an out-of-bounds error. We exclude the bitmask from the `bitmask_and` operation. - Scalar is null; implies it is null across the base column's size. This would make all the output rows null. The scalar's bitmask is also of size 1. In this case, we output a null-mask of the base column's size, and mark all the elements as null. Follows up on rapidsai#18023, rapidsai#18820 Authors: - Basit Ayantunde (https://github.com/lamarrr) Approvers: - David Wendt (https://github.com/davidwendt) - Tianyu Liu (https://github.com/kingcrimsontianyu) - Muhammad Haseeb (https://github.com/mhaseeb123) - Nghia Truong (https://github.com/ttnghia) URL: rapidsai#18845
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This merge request implements null-handling for Transforms. The implementation outputs a null row if any of its input rows are null. The output bitmask is computed in a single pass using the null-masks of the input columns before the transform function is run on the valid output rows.
It also handles the scalar input edge cases:
Scalar is not null; implies it is valid across the base column's size. We also can't use its bitmask as it is a bitmask of size 1, otherwise, we'd encounter an out-of-bounds error. We exclude the bitmask from the
bitmask_andoperation.Scalar is null; implies it is null across the base column's size. This would make all the output rows null. The scalar's bitmask is also of size 1. In this case, we output a null-mask of the base column's size, and mark all the elements as null.
Follows up on #18023, #18820
Checklist