Cannot import autosklearn.classification with issue 'undefined symbol: PyFPE_jbuf'

[RedLiN5]

Hi!
I have heard of lots benefit of auto-sklearn, so recently I also tried to install and use it.

I installed everything under anaconda3, and my Python 3.5 works well on everything but auto-sklearn.

My annoying issues are posted below. I really appreciate it if anyone could fix those for me!

 leslie@leslie-VirtualBox:~$which python 
/home/leslie/anaconda3/bin/python

which pip
/home/leslie/anaconda3/bin/pip

leslie@leslie-VirtualBox:~$ pip -V
pip 8.1.2 from /home/leslie/anaconda3/lib/python3.5/site-packages (python 3.5)

leslie@leslie-VirtualBox:~$ gcc --version
gcc (Ubuntu 5.4.1-2ubuntu1~16.04) 5.4.1 20160904

leslie@leslie-VirtualBox:~$ g++ --version
g++ (Ubuntu 5.4.1-2ubuntu1~16.04) 5.4.1 20160904

Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
[1]  import autosklearn.classification
/home/leslie/anaconda3/lib/python3.5/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
/home/leslie/anaconda3/lib/python3.5/site-packages/sklearn/grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
  DeprecationWarning)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/auto_sklearn-0.1.0-py3.5-linux-x86_64.egg/autosklearn/classification.py", line 1, in <module>
    from autosklearn.estimators import AutoSklearnClassifier
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/auto_sklearn-0.1.0-py3.5-linux-x86_64.egg/autosklearn/estimators.py", line 7, in <module>
    import autosklearn.automl
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/auto_sklearn-0.1.0-py3.5-linux-x86_64.egg/autosklearn/automl.py", line 27, in <module>
    from autosklearn.smbo import AutoMLSMBO
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/auto_sklearn-0.1.0-py3.5-linux-x86_64.egg/autosklearn/smbo.py", line 13, in <module>
    from smac.smbo.smbo import SMBO, get_types
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/smac/smbo/smbo.py", line 15, in <module>
    from smac.epm.rf_with_instances import RandomForestWithInstances
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/smac/epm/rf_with_instances.py", line 4, in <module>
    import pyrfr.regression
ImportError: /home/leslie/anaconda3/lib/python3.5/site-packages/pyrfr/regression.cpython-35m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf

[mfeurer]

Hi,

Please excuse the trouble installing auto-sklearn. We don't yet have any solution for this problem, but in issue #155 they're building a docker image which should alleviate the mentioned problem.

The basic problem is the following: https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/cB2lmykcxMI

I also just figured out that anaconda actually has a package gcc, which at least in my virtual machine fixes the problem you encountered. Try conda install gcc prior to installing auto-sklearn. If you want to reinstall auto-sklearn in the same conda environment, make sure to clean your pip wheels cache and uninstall the package pyrfr. Let me know whether this works.

[RedLiN5]

@mfeurer Thanks for your response!
However, after reinstalling auto-sklearn via pip --no-cache-dir install auto-sklearn.

The error changed to this

>>> import autosklearn.classification
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/leslie/Desktop/auto-sklearn-master/autosklearn/classification.py", line 1, in <module>
    from autosklearn.estimators import AutoSklearnClassifier
  File "/home/leslie/Desktop/auto-sklearn-master/autosklearn/estimators.py", line 7, in <module>
    import autosklearn.automl
  File "/home/leslie/Desktop/auto-sklearn-master/autosklearn/automl.py", line 27, in <module>
    from autosklearn.smbo import AutoMLSMBO
  File "/home/leslie/Desktop/auto-sklearn-master/autosklearn/smbo.py", line 13, in <module>
    from smac.smbo.smbo import SMBO, get_types
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/smac/smbo/smbo.py", line 15, in <module>
    from smac.epm.rf_with_instances import RandomForestWithInstances
  File "/home/leslie/anaconda3/lib/python3.5/site-packages/smac/epm/rf_with_instances.py", line 4, in <module>
    import pyrfr.regression
ImportError: /home/leslie/anaconda3/lib/python3.5/site-packages/scipy/special/../../../../libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/leslie/anaconda3/lib/python3.5/site-packages/pyrfr/regression.cpython-35m-x86_64-linux-gnu.so)

This one has been solved by export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6

The issue still comes back to my question, thanks

[mfeurer]

Sorry, I'm not sure from your answer whether the problem is actually solved. Not finding libstdc++ seem to be a symptom of using the wrong gcc compiler (see here). Could you please execute the following commands and check whether they help:

export LD_PRELOAD=
conda install gcc
which gcc
pip uninstall pyrfr
pip install --no-cache-dir -v pyrfr

If auto-sklearn still fails, could you please paste the output of the pip install command? Please don't try only reinstalling auto-sklearn. The problem you're facing here is a problem with one of it's dependencies. It was already built and installed, that's why it probably still fails.

[RedLiN5]

I really appreciate your help!!! @mfeurer
The solution is pip install --no-cache-dir -v pyrfr

Thanks a lot! lol

[mfeurer]

@sfalkner it seems like this is the solution to the pyrfr + gcc + anaconda problem :)