You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug $.'a is a valid path from Spark's check, so it is normalized to $[''a'] and passed to kernel in plugin, resulting in
ai.rapids.cudf.CudfException: CUDF failure on: /home/jenkins/agent/workspace/jenkins-spark-rapids-jni_nightly-dev-690-cuda11/thirdparty/cudf/cpp/src/json/json_path.cu:631: Invalid empty name in JSONPath query string
Note that $[''a'] is actually an invalid path from Spark's validation. So maybe the normalization we did in plugin b9c292c is losing some information when converting the List[PathInstruction] back to a string and causing some problems.
And at the same time, cuDF kernel can't match this case as well if we remove the normalization.
data: {"'a":"1"}
Spark, spark-rapids, cudf and jsonpath.com will give very different results in this case.
JSONPath
Spark
spark-rapids
cudf kernel(spark-rapids without normalization)
jsonpath.com
$[''a']
null
null
Invalid empty name in JSONPath query string
["1"]
$.'a
1
Invalid empty name in JSONPath query string
Encountered invalid JSONPath input string
No match
I think we can fallback on this case until we have a path parser that matches Spark's behavior.
The text was updated successfully, but these errors were encountered:
Describe the bug
$.'a
is a valid path from Spark's check, so it is normalized to$[''a']
and passed to kernel in plugin, resulting inNote that
$[''a']
is actually an invalid path from Spark's validation. So maybe the normalization we did in plugin b9c292c is losing some information when converting theList[PathInstruction]
back to a string and causing some problems.And at the same time, cuDF kernel can't match this case as well if we remove the normalization.
data:
{"'a":"1"}
Spark, spark-rapids, cudf and jsonpath.com will give very different results in this case.
I think we can fallback on this case until we have a path parser that matches Spark's behavior.
The text was updated successfully, but these errors were encountered: