Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] case when on some nested types can produce a crash #3139

Closed
revans2 opened this issue Aug 4, 2021 · 6 comments · Fixed by #3296
Closed

[BUG] case when on some nested types can produce a crash #3139

revans2 opened this issue Aug 4, 2021 · 6 comments · Fixed by #3296
Assignees
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf P0 Must have for release

Comments

@revans2
Copy link
Collaborator

revans2 commented Aug 4, 2021

Describe the bug

Caused by: ai.rapids.cudf.CudfException: cuDF failure at: /home/roberte/src/cudf/cpp/src/reshape/interleave_columns.cu:38: Called `interleave_columns` on none-supported data type.
E                   	at ai.rapids.cudf.ColumnVector.makeList(Native Method)
E                   	at ai.rapids.cudf.ColumnVector.makeList(ColumnVector.java:441)
E                   	at ai.rapids.cudf.ColumnVector.makeList(ColumnVector.java:410)
E                   	at ai.rapids.cudf.ColumnVector.empty(ColumnVector.java:1623)
E                   	at ai.rapids.cudf.ColumnVector.empty(ColumnVector.java:1630)
E                   	at ai.rapids.cudf.Scalar.listFromNull(Scalar.java:357)
E                   	at com.nvidia.spark.rapids.GpuScalar$.from(literals.scala:221)
E                   	at com.nvidia.spark.rapids.GpuScalar.getBase(literals.scala:442)
E                   	at com.nvidia.spark.rapids.GpuConditionalExpression.$anonfun$computeIfElse$3(conditionalExpressions.scala:42)

This happens when you do a case when statement and no default value is given. The types involved are.

Array(Struct(key: String(not_null), value: Array(Struct(key: String(not_null), value: String))))

This is the same as a

Map(String(not_null),Map(String(not_null),String))

but we don't support maps on case when yet. I was trying to add them and ran into this.

@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Aug 4, 2021
@revans2
Copy link
Collaborator Author

revans2 commented Aug 4, 2021

This can also impacts GpuCoalesce and GpuLiteral.

@Salonijain27 Salonijain27 added cudf_dependency An issue or PR with this label depends on a new feature in cudf P0 Must have for release and removed ? - Needs Triage Need team to review and classify labels Aug 10, 2021
@revans2
Copy link
Collaborator Author

revans2 commented Aug 10, 2021

I filed rapidsai/cudf#9010 to fix this in cudf, but because this is a feature request (cudf does not claim to support this operation) we need to fix this in the short term by either disabling this in the plugin until we can make it work, or find an alternative way to do something in the cudf java APIs for null columns.

@firestarman
Copy link
Collaborator

firestarman commented Aug 11, 2021

Seems this is similar to rapidsai/cudf#8927. And there is a relevant PR rapidsai/cudf#9012.

@revans2
Copy link
Collaborator Author

revans2 commented Aug 11, 2021

Yup I duplicated rapidsai/cudf#9010 to rapidsai/cudf#8927. github search is just not as good as I would like it to be.

@ttnghia
Copy link
Collaborator

ttnghia commented Aug 20, 2021

So should we close this?

@revans2
Copy link
Collaborator Author

revans2 commented Aug 20, 2021

So should we close this?

After we add in tests to verify that it is working and does not break again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf P0 Must have for release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants