[REVIEW] Fix bug handling multi-index empty groupby and add multi-index slicing #1659
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…into bug-ext-cudf-empty-groupby
…e to multiindex.py
Fea/cudf empty groubpy cont
ayushdg
commented
May 8, 2019
ayushdg
changed the title
shwina
reviewed
May 8, 2019
| # unprefix columns | ||
| new_cols = [] | ||
| for c in result.columns: | ||
| new_col = c.split('_')[1] # sum_z-> (sum, z) |
There was a problem hiding this comment.
Perhaps to accommodate column names like person_name, we can do instead:
new_col = c.split('_', 1)[1]
This will split 'sum_person_name' into ['sum', 'person_name']
There was a problem hiding this comment.
Yes! That is my mistake. I'll fix shortly
There was a problem hiding this comment.
I will make that correction in my upcoming PR.
thomcom
reviewed
May 8, 2019
|
|
||
| # converting levels to numpy array will produce a Float64Index | ||
| # (on empty levels)for levels mimicking the behavior of Pandas | ||
| self.levels = np.array(levels) |
There was a problem hiding this comment.
This ended up being a significant performance regression. See the fix in #1689. :D
thomcom
approved these changes
May 8, 2019
shwina
approved these changes
May 8, 2019
|
🔥 |
|
Also fixes rapidsai/dask-cudf#231 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pr is a combination of the work here: #1639 + adding the ability to slice multiiindex objects.
This should make the multi index groupby more compatible with dask groupbys on Dask_cudf dataframes.
Thank you @thomcom for the empty Dataframe updates for groupby. I have used most of them as is from your PR