Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Support sorting and shuffle of decimal #1328

Closed
revans2 opened this issue Dec 8, 2020 · 2 comments · Fixed by #1392
Closed

[FEA] Support sorting and shuffle of decimal #1328

revans2 opened this issue Dec 8, 2020 · 2 comments · Fixed by #1392
Assignees
@revans2
Labels
feature request New feature or request
Milestone
Dec 7 - Dec 18

Comments

@revans2
Copy link
Collaborator

revans2 commented Dec 8, 2020

Is your feature request related to a problem? Please describe.
CUDF supports this so we should enable it and test it. This is likely going to involve enabling this for both SortExec and SortOrder. We may want to also verify that RangePartitioning works properly in this case.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 8, 2020
@revans2 revans2 mentioned this issue Dec 8, 2020
Closed
27 tasks
@revans2 revans2 self-assigned this Dec 10, 2020
@revans2 revans2 changed the title [FEA] Support sorting decimal [FEA] Support sorting and shuffle of decimal Dec 10, 2020
@revans2
Copy link
Collaborator Author

revans2 commented Dec 10, 2020

Moved shuffle here because a lot of the integration tests rely on both sort and shuffle working

@revans2
Copy link
Collaborator Author

revans2 commented Dec 11, 2020

Sorting works but DECIMAL64 appears to get some data corruption when hash_partition is used. I filed rapidsai/cudf#6996 for this.

@revans2 revans2 mentioned this issue Dec 11, 2020
Merged
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Dec 15, 2020
@sameerz sameerz added this to the Dec 7 - Dec 18 milestone Dec 16, 2020
@revans2 revans2 mentioned this issue Dec 16, 2020
Merged
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to 9d794877fdb7822ea2194e045aba45e0317cf577 (NV…
b578c96 
…IDIA#1328)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

Labels
feature request New feature or request
Projects
None yet
Milestone
Dec 7 - Dec 18
Development

Successfully merging a pull request may close this issue.

Turn on hash partitioning for decimal support revans2/spark-rapids
2 participants
@revans2 @sameerz