Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better GPU Cast type checks #1402

Merged
merged 2 commits into from
Dec 16, 2020
Merged

Conversation

revans2
Copy link
Collaborator

@revans2 revans2 commented Dec 15, 2020

This automates generating the docs for cast. It also updates the tests in cast so that they go off of the types listed as supported.

I did take one of the changes from #1396 to make this work (Thanks @andygrove for that work)

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2 revans2 added this to the Dec 7 - Dec 18 milestone Dec 15, 2020
@revans2 revans2 self-assigned this Dec 15, 2020
@revans2
Copy link
Collaborator Author

revans2 commented Dec 15, 2020

build

@github-actions
Copy link

👎 Promotion blocked, new vulnerability found

Vulnerability report

Component Vulnerability Description Severity
Guava: Google Core Libraries for Java CVE-2020-8908 A temp directory creation vulnerability exist in Guava versions prior to 30.0 allowing an attacker with access to the machine to potentially access data in a temporary directory created by the Guava com.google.common.io.Files.createTempDir(). The permissions granted to the directory created default to the standard unix-like /tmp ones, leaving the files open. We recommend updating Guava to version 30.0 or later, or update to Java 7 or later, or to explicitly change the permissions after the creation of the directory if neither are possible. LOW

"2018-1random_text",
"2018-11-08random_text",
"2018-11-9random_text",
// `yyyy-[m]m-[d]dT*` in some version of Spark this is valid and in others it is not
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the only time we are excluding these values from the test is when ansiEnabled=true rather than excluding them for particular Spark versions?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, version 3.1 + when ANSI is enabled they now cause an exception to be thrown. Because the test this is for is "Test all supported casts with in-range values" I assumed that is fine. If you want me to allow different values for different versions I can.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is good but the comment confused me so I just wanted to make sure that I understood this correctly.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update the comment to explain what is happening.

@pxLi
Copy link
Collaborator

pxLi commented Dec 16, 2020

build

@revans2 revans2 merged commit b35e7ca into NVIDIA:branch-0.4 Dec 16, 2020
@revans2 revans2 deleted the cast_type_docs branch December 16, 2020 14:23
@sameerz sameerz added documentation Improvements or additions to documentation test Only impacts tests labels Dec 26, 2020
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
…IDIA#1402)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation test Only impacts tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] CastOpSuite and AnsiCastOpSuite failing with ArithmeticException on Spark 3.1
4 participants