Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy functions don't treat Any[...] arrays like Python lists #486

Closed
stevengj opened this issue Mar 28, 2024 · 2 comments
Closed

numpy functions don't treat Any[...] arrays like Python lists #486

stevengj opened this issue Mar 28, 2024 · 2 comments

Comments

@stevengj
Copy link
Member

A problem reported in JuliaPy/PythonPlot.jl#36 — Numpy functions work for empty Python lists, and for empty numeric-typed Julia arrays, but don't work for empty Any[] arrays:

julia> using PythonCall

julia> np = pyimport("numpy")
Python: <module 'numpy' from '/Users/stevenj/.julia/environments/v1.10/.CondaPkg/env/lib/python3.12/site-packages/numpy/__init__.py'>

julia> np.isfinite(Int[])
Python: array([], dtype=bool)

julia> np.isfinite(pylist())
Python: array([], dtype=bool)

julia> np.isfinite([])
ERROR: Python: TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Python stacktrace: none
Stacktrace:
 [1] pythrow()
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/err.jl:94
 [2] errcheck
   @ ~/.julia/packages/PythonCall/wXfah/src/err.jl:10 [inlined]
 [3] pycallargs
   @ ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:210 [inlined]
 [4] pycall(f::Py, args::Vector{Any}; kwargs::@Kwargs{})
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:228
 [5] pycall
   @ ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:218 [inlined]
 [6] (::Py)(args::Vector{Any})
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/Py.jl:341
 [7] top-level scope
   @ REPL[5]:1

Ideally, Py([]) should behave much like a python empty list. I'm not sure if this is something that can be fixed on the PythonCall side or if it's some broken special-casing in numpy?

The error message stems from linear_search_type_resolver in numpy

The basic issue seems to be that numpy treats an empty list as an array of float64, but Py([]) as an array of object:

julia> np.asarray(pylist())
Python: array([], dtype=float64)

julia> np.asarray([])
Python: array([], dtype=object)

The latter seems more correct, but I'm guessing that Numpy implemented the former rule for convenience. I can't find where the former rule is implemented in the numpy source, however, to tell if we can hook into it.

Not sure if this is something that can/should be changed, but I thought I should make a record of the problem for future reference.

@stevengj stevengj changed the title numpy functions don't treat Py([]) like Python [] == pylist() numpy functions don't treat Any[...] arrays like Python lists Mar 28, 2024
@stevengj
Copy link
Member Author

It's not just empty arrays:

julia> np.isfinite(pylist([1,2,3]))
Python: array([ True,  True,  True])

julia> np.isfinite(Any[1,2,3])
ERROR: Python: TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

julia> np.isfinite(np.asarray(pylist([1,2,3]), dtype=pybuiltins.object))
ERROR: Python: TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Basically, NumPy seems to treat Python lists fundamentally differently from numpy arrays of objects, and Py([]) acts like the latter. Seems like an upstream issue.

@stevengj
Copy link
Member Author

I posted a question on https://mail.python.org/archives/list/numpy-discussion@python.org/ … we'll see what they say.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant