Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] we need to mark columns from received shuffle buffers as GpuColumnVectorFromBuffer #860

Closed
abellina opened this issue Sep 26, 2020 · 0 comments · Fixed by #872
Closed

[BUG] we need to mark columns from received shuffle buffers as GpuColumnVectorFromBuffer #860

abellina opened this issue Sep 26, 2020 · 0 comments · Fixed by #872
Assignees
@abellina
Labels
bug Something isn't working performance A performance related task/issue shuffle things that impact the shuffle plugin
Milestone
Sep 28 - Oct 9

Comments

@abellina
Copy link
Collaborator

abellina commented Sep 26, 2020 •
edited
Loading

Batches built from the UCX shuffle could have non GpuColumnVectorFromBuffer columns, and this causes extra contig_split calls within the spillable coalesce added #763.

I need to spend a bit more time here, but I think the fix is simple and similar to: #851, that way when we add batches in the spillable coalesce, we don't call contig_split, for every single received batch.

I am working on a fix, but will be in sometime Sunday or Monday.

@revans2 @jlowe fyi.
@abellina abellina added bug Something isn't working ? - Needs Triage Need team to review and classify labels Sep 26, 2020
@abellina abellina self-assigned this Sep 26, 2020
@abellina abellina added shuffle things that impact the shuffle plugin performance A performance related task/issue labels Sep 26, 2020
@abellina abellina added this to the Sep 28 - Oct 9 milestone Sep 28, 2020
@abellina abellina mentioned this issue Sep 28, 2020
Merged
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Sep 29, 2020
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to 084555e7c1d25e99267edc9afc3c831fbec24100 (NV…
e1c998e 
…IDIA#860)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abellina abellina

Labels
bug Something isn't working performance A performance related task/issue shuffle things that impact the shuffle plugin
Projects
None yet
Milestone
Sep 28 - Oct 9
Development

Successfully merging a pull request may close this issue.

Ensure that receive-side batches use GpuColumnVectorFromBuffer to avoid abellina/spark-rapids
2 participants
@abellina @sameerz