Mac OS Sundials bug

[valentinsulzer]

Descirbe the bug
scikits.odes does not work ("image not found") when following installation instructions

Explanation
As I understand it the problem is that scikits.odes looks for libsundials_sunlinsolklu.3.dylib, but brew install sundials installs a more recent version, which only has

libsundials_sunlinsolklu.dylib
libsundials_sunlinsollapackband.3.5.0.dylib

so libsundials_sunlinsolklu.3.dylib cannot be found.

This is not caught by the Mac OS tests since in CI the Mac OS tests use the script install_KLU_Sundials.py

Furthermore, if installing locally, with tox -e pybamm-requires, the script is used (which installs the correct version of sundials) but only LD_LIBRARY_PATH is updated, and not DYLD_LIBRARY_PATH as should be the case for mac.
This means that scikits.odes is still looking in /usr/local/opt/ for sundials, where it finds the incorrect brew installation, instead of in ~/.local/lib

** Solution **

• Remove the requirement for brew install sundials from docs
• Use the KLU script to install sundials on mac, and also update DYLD_LIBRARY_PATH

This could also be an opportunity to clean up the install docs, which I still find a little bit confusing

[tlestang]

I couldn't reproduce this, for instance this GitHub workflow installs sundials using homebrew and runs the tests for the scikits.odes solver fine.

Using Homebrew is also preferable for compiling the idaklu extension. When compiling Sundials ans SuiteSparse manually, the path to the suitesparse components ends up hardcoded in the idaklu.dylib, ignoring the [DY]LD_LIBRARY_PATH. It's fine if you use the same libraries that you compiled the extension with, but this is not generally the case - for instance for a user that downloads the pre-compiled wheel and installs sundials using Homebrew. I remember pulling my hair out on this during the summer, and coming to the conclusion that using Homebrew solves the issue, and works both for the idaklu extension and for installing/using scikits.odes.

You're right that the CI workflow should use Homebrew instead of the KLU script on macos. I suspect it's a simple oversight when switching to using tox in the workflow.

[shday]

I think I have the same issue.

I just installed pybamm in a fresh venv on my Mac. Used "pip install pybamm". (Note that I never did "brew install sundials" as mentioned by @tinosulzer )

The install seemed to work fine, but then I get an error when importing pybamm:

>>> import pybamm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/pybamm/__init__.py", line 231, in <module>
    from .solvers.idaklu_solver import IDAKLUSolver, have_idaklu
  File "/Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/pybamm/solvers/idaklu_solver.py", line 13, in <module>
    idaklu = importlib.util.module_from_spec(idaklu_spec)
ImportError: dlopen(/Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so, 2): Library not loaded: /usr/local/opt/sundials/lib/libsundials_ida.5.dylib
  Referenced from: /Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so
  Reason: image not found

I'm not sure what to do to get it working.

FYI, here's what pip reported after the install:

Installing collected packages: six, anytree, kiwisolver, cycler, python-dateutil, pyparsing, pillow, numpy, matplotlib, casadi, scipy, meshio, scikit-fem, flatbuffers, absl-py, jaxlib, opt-einsum, jax, tornado, ipython-genutils, traitlets, pyzmq, jupyter-core, jupyter-client, backcall, ptyprocess, pexpect, appnope, wcwidth, prompt-toolkit, pickleshare, parso, jedi, decorator, pygments, ipython, ipykernel, qtpy, qtconsole, terminado, Send2Trash, attrs, pyrsistent, jsonschema, nbformat, prometheus-client, MarkupSafe, jinja2, pycparser, cffi, argon2-cffi, jupyterlab-pygments, pandocfilters, defusedxml, testpath, entrypoints, mistune, async-generator, nest-asyncio, nbclient, packaging, webencodings, bleach, nbconvert, notebook, widgetsnbextension, ipywidgets, jupyter-console, jupyter, future, autograd, pytz, pandas, pybamm
  Running setup.py install for jax ... done
  Running setup.py install for pyrsistent ... done
  Running setup.py install for pandocfilters ... done
  Running setup.py install for future ... done
  Running setup.py install for autograd ... done
Successfully installed MarkupSafe-1.1.1 Send2Trash-1.5.0 absl-py-0.11.0 anytree-2.8.0 appnope-0.1.2 argon2-cffi-20.1.0 async-generator-1.10 attrs-20.3.0 autograd-1.3 backcall-0.2.0 bleach-3.2.1 casadi-3.5.5 cffi-1.14.4 cycler-0.10.0 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 flatbuffers-1.12 future-0.18.2 ipykernel-5.4.2 ipython-7.19.0 ipython-genutils-0.2.0 ipywidgets-7.5.1 jax-0.2.5 jaxlib-0.1.57 jedi-0.17.2 jinja2-2.11.2 jsonschema-3.2.0 jupyter-1.0.0 jupyter-client-6.1.7 jupyter-console-6.2.0 jupyter-core-4.7.0 jupyterlab-pygments-0.1.2 kiwisolver-1.3.1 matplotlib-3.3.3 meshio-4.3.8 mistune-0.8.4 nbclient-0.5.1 nbconvert-6.0.7 nbformat-5.0.8 nest-asyncio-1.4.3 notebook-6.1.5 numpy-1.19.4 opt-einsum-3.3.0 packaging-20.8 pandas-1.1.5 pandocfilters-1.4.3 parso-0.7.1 pexpect-4.8.0 pickleshare-0.7.5 pillow-8.0.1 prometheus-client-0.9.0 prompt-toolkit-3.0.8 ptyprocess-0.6.0 pybamm-0.3.0.post1 pycparser-2.20 pygments-2.7.3 pyparsing-2.4.7 pyrsistent-0.17.3 python-dateutil-2.8.1 pytz-2020.4 pyzmq-20.0.0 qtconsole-5.0.1 qtpy-1.9.0 scikit-fem-2.3.0 scipy-1.5.4 six-1.15.0 terminado-0.9.1 testpath-0.4.4 tornado-6.1 traitlets-5.0.5 wcwidth-0.2.5 webencodings-0.5.1 widgetsnbextension-3.5.1
WARNING: You are using pip version 19.2.3, however version 20.3.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[valentinsulzer]

Thanks for the extra information.
If you look in /usr/local/opt/sundials/lib/, my guess is you won't find exactly libsundials_ida.5.dylib but some other version.

Can you see if this fixes it:

1. open your activate file in your virtualenv (venv/bin/activate) or tox (.tox/dev/bin/activate)
2. make a copy of the line that starts with LD_LIBRARY_PATH but call it DYLD_LIBRARY_PATH instead. It should be something like export LD_LIBRARY_PATH=/Users/vsulzer/.local/lib:. This should tell it to look for the sundials you installed in your .local/lib folder instead of in /usr/local/opt
3. re-active (source path/to/activate)

[shday]

Thanks for the responding.

Unfortunately, I don't even have a /usr/local/opt/sundials/ folder. I also didn't have anything like LD_LIBRARY_PATH in my activate file. Perhaps I used a different way to create my virtual environment? (https://docs.python.org/3/library/venv.html)

I did try running the install_KLU_Sundials.py script, but got an error that CMake must be installed. I'll probably try that next.

[valentinsulzer]

Ah I see you are installing pybamm via pip now, which should mean you don't have access to the IDAKLU solver. But weirdly, here it is finding an idaklu_spec, which might be from a different project you have using sundials? Or an old sundials install? I'm not sure exactly how importlib looks for these things. I feel like idaklu.cpython-38-darwin.so shouldn't exist, but I don't know why it got created or whether you can safely remove it

[tlestang]

Hi @shday and @tinosulzer, I sense there is a bit of confusion here, which I hope I can clear.

Short answer

If you're on MacOS, you must install the sundials library yourself. There are several ways to do that, but the most straightforward is to use Homebrew, following the instructions available here. I suggest you give it a try and tell us the result.

Longer answer:

Whenever you type pip install pybamm, pip fetches the PyBaMM package from the Python Package Index (PyPI). However, pip does not download all python files one by one, but rather an archive that contains the PyBaMM's source code, some metadata and compiled modules, like the idaklu module. Indeed, if you look into PyBaMM's sources, you'll find that the idaklu module is actually written in C++, and must therefore be compiled before being available for import. As a user, you don't have to do that yourself though because the module was compiled by us before hand, and the resulting idaklu.cpython-38-darwin.so file is included in the archive pip downloads and installs.
However, the idaklu module has a runtime dependency on the sundials library, that must be available (in the form of a shared library) on your system when you import the module. For GNU/Linux systems, the compiled version of sundials is also included in the archive pip downloads, therefore a pip install pybamm is all that is required to get everything running. On Mac however, the compiled sundials is not included in that archive, and therefore users must install sundials themselves. Thanks to Homebrew, that's just another short command.

Now, you could also choose to clone the PyBaMM repo and install pybamm from that instead of the archive available on PyPI. This would be a good choice if you wanted to e.g. modify/contribute to PyBaMM. In this case however, you don't benefit from the pre-compiled modules and dependencies, and you'll have to do it yourself. As you've noticed already, the PyBaMM repo contains a python script that automatically downloads, compile and install the sundials library (install_KLU_Sundials.py). The idaklu module is automatically compiled when you invoke pip on the PyBaMM repo. The compilation and installation of PyBaMM from the repo is described here.

I hope that's helpful - both in practice as well as to help understand the different moving parts.
Please let us know if youŕe confused by the installations instructions: we're about to refine them to hopefully make them clearer, and any feedback is useful.

[valentinsulzer]

Ahh that clears some things up, thanks @tlestang !

[shday]

Thanks for the explanation.

I tried brew install sundials and it seems to try to install Python3.9 (there were several errors). I think I'll try installing 3.9 separately first as I'm still using 3.8.

[valentinsulzer]

Unfortunately pybamm isn't guaranteed to work with python 3.9 yet as we are still waiting for some dependencies to upgrade. Testing now to see if it works anyway: https://github.com/pybamm-team/PyBaMM/runs/1655551286

[JLinx44]

Hi all, I get the same error as @shday when importing pybamm. I did the brew install sundials and the pip install pybamm, both of which seemed to have installed fine. I am very new to working with Python and did not install it in a virtual environment, which could be the cause of my issue. From @tinosulzer comment about a potential fix, I will say that when looking for the libsundials_ida.5.dylib file I did in fact find it in the /usr/local/opt/sundials/lib/ directory. I am not sure if this is anything useful, but I figured I should offer something beyond simply saying I get the same error.

[valentinsulzer]

So you get an error like

ImportError: dlopen(/Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so, 2): Library not loaded: /usr/local/opt/sundials/lib/libsundials_ida.5.dylib
  Referenced from: /Users/stephenday/projects/tinypico/venv/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so
  Reason: image not found

but that exact file does exist?

Can you try the DYLD_LIBRARY_PATH fix suggested above?

[JLinx44]

@tinosulzer I was mistaken that I had the exact same error as @shday, my error is:

ImportError: dlopen(/opt/anaconda3/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so, 2): Library not loaded: /usr/local/opt/sundials/lib/libsundials_sunlinsolklu.3.5.0.dylib Referenced from: /opt/anaconda3/lib/python3.8/site-packages/idaklu.cpython-38-darwin.so Reason: image not found

I looked for the libsundials_sunlinsolklu.3.5.0.dylib file and did not find it exactly but found libsundials_sunlinsolklu.3.6.1.dylib as you suspected.

The only activate file I found was not in a bin folder but in a scripts folder buried deep in a mess of versions and frameworks, this along with the fact that it did not have a line similar to export LD_LIBRARY_PATH=/Users/vsulzer/.local/lib: or any line with LD_LIBRARY_PATH like Stephen also reported. I am not sure if I should be looking in another directory, or if you mean I should be editing one of the "Unix executable" files, which I am also not sure how to do. So I did not really try that fix that you suggested. My apologies for being so inexperienced in this and for not responding until now.

[valentinsulzer]

Did you use a virtual environment, conda, or tox to install?

[JLinx44]

I believe I used conda because I set up my python space by downloading Anaconda. Also did not create a virtual environment and am not familiar with tox.

[valentinsulzer]

I'm not sure how to help if you didn't use a virtual environment. Could you try again using a virtual environment?