You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Cudf supports only strf patterns while spark supports Java date/time patterns defined in SimpleDateFormat as of Spark 3.0. We have a utility function in DateUtils that converts a Java date/time pattern literal to strf pattern. We need a way to support patterns that are vectors
Describe the solution you'd like
It may be possible for us to write a unary transform to convert a Java date/time vector to a strf pattern vector
Describe alternatives you've considered
Currently there are no viable alternatives, unless cudf decides to support Java patterns.
The text was updated successfully, but these errors were encountered:
We need a way to support patterns that are vectors
To clarify, the pattern vector is a column of patterns that has the same row count as the string column to be parsed, i.e.: each row can have a custom format string describing how the timestamp should be parsed.
tgravescs
pushed a commit
to tgravescs/spark-rapids
that referenced
this issue
Nov 30, 2023
Is your feature request related to a problem? Please describe.
Cudf supports only strf patterns while spark supports Java date/time patterns defined in SimpleDateFormat as of Spark 3.0. We have a utility function in DateUtils that converts a Java date/time pattern literal to strf pattern. We need a way to support patterns that are vectors
Describe the solution you'd like
It may be possible for us to write a unary transform to convert a Java date/time vector to a strf pattern vector
Describe alternatives you've considered
Currently there are no viable alternatives, unless cudf decides to support Java patterns.
The text was updated successfully, but these errors were encountered: