Support reading decimal columns from parquet files

[sperlingxx]

This pull request is to enable reading decimal columns from parquet files via turning allowDecimal to true. This pull request also provides test coverage for decimal reading.
But there exist some limits on decimal reading: we can only read decimal columns whose storage type is INT32/64. For now, cuDF doesn't support FIXED_LENGTH_BYTE_ARRAY. INT32 will be read as INT64 because we only support DECIMAL64 in spark-rapids.

[sperlingxx]

build

[sperlingxx]

build

[sperlingxx]

build

[sperlingxx]

build

[sperlingxx]

build

[sperlingxx]

build

[revans2]

@sameerz @jlowe @tgravescs I personally feel that this is getting to be way too late to put this into the 0.3 release. We really should be looking at bug fixes instead of new feature work. Especially because we know that rapidsai/cudf#6909 is still an outstanding issue. But if this is something we need to get in, then we need to at a minimum we need to throw an exception if we get back a data type we don't expect for a decimal column.

[jlowe]

I personally feel that this is getting to be way too late to put this into the 0.3 release.

Agree. I would like to see the legacy decimal encoding supported before this goes in, otherwise we're left in a situation where the plugin crashes a query that used to work without it.

[sperlingxx]

build

[sperlingxx]

build

[jlowe]

@sperlingxx per the above discussion, I think this PR should be retargeted to branch-0.4.

[sperlingxx]

@sperlingxx per the above discussion, I think this PR should be retargeted to branch-0.4.

Yes! So, I labeled this pull request with WIP.

[jlowe]

Yes! So, I labeled this pull request with WIP.

That's fine, adding a request to retarget this PR to branch-0.4 so it cannot be accidentally merged in the interim.

[sperlingxx]

build

[sperlingxx]

build

[jlowe]

Test failures are related, I'm guessing because the 3.1.0 shims were not updated.

[sperlingxx]

build

[sperlingxx]

build

[jlowe]

I also noticed that this PR didn't generate a new docs/supported_ops.md as it should. We're not currently generating the documentation for scans from the ScanMeta, but it is hardcoded in SupportedOpsDocs.help which should be updated accordingly. After updating the help method, performing a usual mvn verify build will generate a new supported_ops.md file that should be added to this PR.

[sperlingxx]

build

[jlowe]

This is looking better, but the supported_ops.md still states at the bottom of the page that Parquet does not support reading the Decimal type. The static table in TypeChecks needs to be updated to reflect that decimal is support for Parquet input and the supported_ops.md file regenerated by mvn verify.

[sperlingxx]

build

[sperlingxx]

This is looking better, but the supported_ops.md still states at the bottom of the page that Parquet does not support reading the Decimal type. The static table in TypeChecks needs to be updated to reflect that decimal is support for Parquet input and the supported_ops.md file regenerated by mvn verify.

The static table has been updated.