Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Add in std::overflow handling in CUDF and plugin #8451

Open
revans2 opened this issue May 31, 2023 · 0 comments
Open

[FEA] Add in std::overflow handling in CUDF and plugin #8451

revans2 opened this issue May 31, 2023 · 0 comments
Labels
feature request New feature or request reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented May 31, 2023

Is your feature request related to a problem? Please describe.
rapidsai/cudf#13323 went into 23.06, which added in support for throwing a std::overflow_error when a column would violate the limits on a CUDF column.

We should add in translations in the CUDF JNI layer so that when we see a std::overflow_error we throw a java exception that corresponds to it.

Then in the plugin when we see this type of an exception we can treat this as something very similar to a split and retry exception. We need to be a bit careful because they are not 100% the same. Some "split" operations may just change config settings instead of changing the actual input data. Also most split operations with a split is no longer possible will throw a SplitAndRetryOOM exception to indicate this. We don't want to do that if it was not actually caused by a SplitAndRetryOOM.

The main places to think about this are probably with a project operation or with aggregations (collect_set/collect_list) like operations. The later is more difficult to make it work though.

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels May 31, 2023
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Development

No branches or pull requests

2 participants