Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Verify that cast works in other timeszones for casts that don't need timezones #6835

Open
revans2 opened this issue Oct 18, 2022 · 0 comments
Labels
feature request New feature or request

Comments

@revans2
Copy link
Collaborator

revans2 commented Oct 18, 2022

Is your feature request related to a problem? Please describe.
This is very closely related to #6832, and if #6832 does everything properly we might not need this issue. Cast currently will fall back to the CPU if we are in any timezone that is not UTC. But not all cast operations actually use the timezone, despite Cast asking for it. we need to be sure that we have updated our code so that cast will only fall back to the CPU if a timezone is set, and the cast operation uses the time zone to get the right answer. i.e. cast short to int does not care at all about timezones. But cast timestamp to string does.

https://github.com/apache/spark/blob/2acfc1dbca975a2a4a38124fe8ebe464aa1663a9/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L261-L284

is what Spark has to solve this problem not sure if we should reuse it or what.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants