Move drop_duplicates, drop_na, _gather, take to IndexFrame and create their _base_index counterparts
#9807
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…dropna_list_interface
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
…ropna_list_interface
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Codecov Report
@@ Coverage Diff @@
## branch-22.02 #9807 +/- ##
================================================
- Coverage 10.49% 10.40% -0.09%
================================================
Files 119 119
Lines 20305 20556 +251
================================================
+ Hits 2130 2139 +9
- Misses 18175 18417 +242
Continue to review full report at Codecov.
|
vyasr
requested changes
Dec 3, 2021
Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>
…eate _copy_typemetadata for MultiIndex
vyasr
requested changes
Dec 7, 2021
Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>
vyasr
approved these changes
Dec 8, 2021
…ent/remove_keep_index
brandon-b-miller
approved these changes
Dec 13, 2021
There was a problem hiding this comment.
connected offline with @isVoid and chatted about this a bit. This LGTM.
…ent/remove_keep_index
…ent/remove_keep_index
…ent/remove_keep_index
…ent/remove_keep_index
…ent/remove_keep_index
…f into improvement/remove_keep_index
|
@vyasr @brandon-b-miller 2 changes since your approval:
|
|
Gained approval from both reviewers offline. Rerun tests. |
|
rerun tests |
1 similar comment
|
rerun tests |
|
@gpucibot merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a follow up of #9558 (Part 1 of 3)
One remaining problem from #9558 is that
Frameis index agnostic, however the above functions needs to perform index-aware operations when building the list of columns to pass to libcudf. For example, to remove duplicates ofBaseIndex, it should only construct the list with all its columns. But in a dataframe, it would need to pass in all data columns plus the index columns, while specifying the indices of the data columns to consider duplicates. This complicates for_gatherwhich supportskeep_indexargument. This PR moves aforementioned functions toIndexedFrames, and create its counterparts in_base_index.A couple noteworthy changes:
l(r)hs_is_indexkeep_indexargument is removed. For internal usage it's more advised to use_gather. (And thus this PR is labeled breaking)