support dataframe protocol (tested with Vaex)

[maartenbreddels]

This allows plotly express to take in any dataframe that supports
the dataframe protocol, see:
https://data-apis.org/blog/dataframe_protocol_rfc/
https://data-apis.org/dataframe-protocol/latest/index.html

Test includes an example with vaex, which should work with
vaexio/vaex#1509
(not yet released)

This is only a POC, I think this needs to wait till Pandas implemented the from_dataframe, and if you'd like to keep this test, would require a Vaex version with above mentioned PR merged and released.

Usage:

Note that this does not speed up any aggregation/processing, although reading from hdf5/arrow/parquet might be faster.

[maartenbreddels]

credits go to @AlenkaF for doing the heavy lifting on the Vaex side (cc @rgommers)

[nicolaskruchten]

Cool, thanks! I'll be happy to merge this once it's a bit more ready :)

[nicolaskruchten]

See also #3901 for an alternative approach

[maartenbreddels]

Since pandas 1.5.0 it has support for the protocol: https://pandas.pydata.org/docs/whatsnew/v1.5.0.html#dataframe-interchange-protocol-implementation

[maartenbreddels]

Not sure if you'd like a vaex dependency for testing, but in case you're ok with it, where should that do?

[nicolaskruchten]

Thanks Maarten! There should be a requirements_optional.txt somewhere that I can add a vaex dependency :)

Once we merge this, we can still offer #3901 as a fallback, for cases where someone has an old pandas or an old vaex without the interchange stuff but, say, a vaex that does still export itself to_pandas, right?

[maartenbreddels]

Absolutely, although we have a to_pandas_df() method

[labanyamukhopadhyay]

Hi @nicolaskruchten Is there an update regarding support for the dataframe exchange protocol? It would be useful for interoperability with Plotly and Modin dataframes!

[alexander-beedie]

As an FYI, this would also enable transparent polars integration/support (closing #3637) as we implemented support for the DataFrame Interchange Protocol back in 0.16.2 (end of Jan/beginning of Feb this year) 📈

[MarcoGorelli]

Is this still active?

If so, I'd strongly suggest setting 2.0.2* as the minimum pandas version to try interchanging from, because there's some pretty basic mistakes in earlier versions:

In [1]: df = pl.DataFrame({'a': [1,2,3]})

In [2]: pd.api.interchange.from_dataframe(df[1:])
Out[2]:
                 a
0                2
1                3
2  125822987010162

(😱 )

*not yet available, but should be out tomorrow

[anmyachev]

If so, I'd strongly suggest setting 2.0.2* as the minimum pandas version to try interchanging from, because there's some pretty basic mistakes in earlier versions:

Hi, 2.0.2 is out.

As an option, one can change the condition so that this protocol is used for those users who have already switched to pandas 2.0.2 and not wait until the plotly's minimum supported version of pandas will be 2.0.2.

from packaging import version
if hasattr(args["data_frame"], "__dataframe__") and version.parse(pd.__version__) >= version.parse("2.0.2"):

[MarcoGorelli]

thanks @anmyachev - yes, sorry, that's what I meant, rather than bumping the minimum pandas version for everything

[anmyachev]

Once we merge this, we can still offer #3901 as a fallback, for cases where someone has an old pandas or an old vaex without the interchange stuff but, say, a vaex that does still export itself to_pandas, right?

@nicolaskruchten using interchange protocol and also having a fallback as in #3901 looks the best option for now, since IIUC interchange protocol doesn't work for Series.

cc @MarcoGorelli

[anmyachev]

@LiamConnors @nicolaskruchten do you have a plan to use interchange protocol in addition to to_pandas method that was added in #3901?

In this case, plotly.py would also be able to accept Modin dataframes.

[nicolaskruchten]

This is probably a good idea still yes, if someone wants to update this PR to implement this fallback :)

[anmyachev]

This is probably a good idea still yes, if someone wants to update this PR to implement this fallback :)

Good! In that case, I'll take care of it, if no one minds :)

[anmyachev]

@nicolaskruchten I made a separate pull request, with the continuation of this work. #4244

[anmyachev]

@nicolaskruchten I guess this PR can be closed?

[alexcjohnson]

Yes, thanks @anmyachev!