[FEA][JSON reader] to support parsing with single quotes

[wbo4958]

This is part of FEA of NVIDIA/spark-rapids#9
We have a JSON file

{'name': 'Reynold Xin'}

CUDF can't parse this file because of

ai.rapids.cudf.CudfException: cuDF failure at: /home/bobwang/work.d/nvspark/cudf/cpp/src/io/json/reader_impl.cu:609: Error determining column names.

Spark can parse it by default.

We expect there is a configure allowSingleQuotes to control this behavior.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[revans2]

By default Spark allows single quotes by default so this is a blocker for us to enable JSON parsing in Spark by default.

[GregoryKimball]

After some discussion it seems we could support this by adding new states for single-quoted field names and single-quoted string values. We would have to increase the size of the state machine which would bring a performance penalty. I'll leave this in backlog for now.

Update: our approach here will be to introduce a quote-normalizing preprocessing step based on a new finite state transducer. Also see #13525

[revans2]

I think this is done now. @GregoryKimball @andygrove do you both agree?

[GregoryKimball]

Closed by #14729. Thank you @shrshi!