Stats missing for dataSkippingStatsColumns when escaping column name

[mattfysh]

Environment

Delta-rs version: 0.19.2

Binding: Python

Environment:

• Cloud provider:
• OS: MacOS
• Other:

---

Bug

What happened:

Stats are not captured for some columns in dataSkippingStatsColumns. If you were to write the table without that option, the same columns will appear in min/max values stats.

In the case below, only stats for "x" are captured. It appears that escaping the column name causes the column to be skipped

However, some columns must be escaped otherwise you get an error, e.g.

Stats column z.z not found in schema

What you expected to happen:

The stats captured the columns specified - in this case all of "x", "y" and "z.z"

How to reproduce it:

import polars as pl

df = pl.DataFrame({"x": [1, 2], "y": [5, 6], "z.z": [3, 4]})

df.write_delta(
    "table",
    delta_write_options={
        "configuration": {"delta.dataSkippingStatsColumns": "x,`y`,`z.z`"}
    },
)

More details:

[mattfysh]

A somewhat related bug - if you specify a column in delta.dataSkippingStatsColumns that is binary you'll get this error, even though the same column works without DSSC config option specified

_internal.DeltaError: Generic DeltaTable error: Stats column binary_column has unsupported type binary

It seems odd that by using the DSSC config option, a lot of things that work normally suddenly break?