Migrate column factories to pylibcudf

[brandon-b-miller]

This PR implements column_factories.hpp using pylibcudf and migrates the cuDF cython to use them cc @vyasr

[vyasr]

FYI @brandon-b-miller we should address the TODO I noted in this commit in this PR.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[wence-]

Structure looks good overall two things:

1. It feels like the fused types for empty column construction can be simpler.
2. It would be great if the datatype conversions didn't rely on quite so many third-party libraries.

[wence-]

question: The pylibcudf DataType object doesn't have a concept of nested lists, but the pyarrow datatype does: for pyarrow, the datatype is List(element_type), whereas for pylibcudf it's List. Does that potentially cause problems here?

[brandon-b-miller]

My understanding is that the "nestedness" is a property of the column and not as much a property of the type, at least from libcudf's perspective.

For the from_arrow case, there's nothing to return except for DataType(type_id.LIST). For to_arrow, we have the column to inspect for the non dtype cases, but not for the dtype only case. Since we can't figure out exactly what kind of pa.List or pa.Struct to return, I chose to error there.

[vyasr]

There simply isn't a 1-1 mapping between pyarrow and pylibcudf data types for nested types because in pylibcudf (read: libcudf) the nested types are encoded in the column as Brandon pointed out. So I don't think there's much else we can do in the from_arrow case.

For the to_arrow case, maybe we should accept a kwarg column that can be inspected in the case of list/struct types. WDYT?

[brandon-b-miller]

Walrus (walri?) abound in my attempt here bffe500

[vyasr]

https://www.quora.com/What-is-the-plural-of-%E2%80%98walrus%E2%80%99-Is-it-%E2%80%98walruses%E2%80%99-or-%E2%80%98walri%E2%80%99-considering-that-the-root-word-ends-in-us

😄

[wence-]

issue: So conversion, much wow.

We would like to get to a state where pylibcudf doesn't depend on arrow, numpy, and, in this case, transitively pandas. Can we not introduce that dependency here by explicitly enumerating the handling of all types?

[vyasr]

We definitely shouldn't have a pandas dependence in pylibcudf. pyarrow dependence should be isolated to the interop module, which should only be loaded conditionally. numpy dependence I was less worried about, but from a quick search the only usage I currently see in pylibcudf is here, so it seems like we might be able to strip that out fairly painlessly too.

This PR adds an extra np.dtype call in column.pyx in place of enumerating. I'm in the same boat as Lawrence and think we should avoid using numpy if it's easy to enumerate. I don't know how painful that will be to maintain for all dtypes though.

[brandon-b-miller]

b3e934c reverts the earlier stab at things and adds two explicit mappings.

[vyasr]

Some small things, but generally looks good now. Thanks!

[vyasr]

https://www.quora.com/What-is-the-plural-of-%E2%80%98walrus%E2%80%99-Is-it-%E2%80%98walruses%E2%80%99-or-%E2%80%98walri%E2%80%99-considering-that-the-root-word-ends-in-us

😄

[wence-]

Tiny niggles, looks great.

[wence-]

/merge