WIP: Pip uninstall mayavi and pysurfer from CircleCI

[GuillaumeFavelier]

Let's see what happens if I remove mayavi and pysurfer from requirements.txt

[larsoner]

I don't think we actually want to take this step until we change the default. And even then we still want them for testing.

The next logical step is just to remove them from CircleCI I think

[GuillaumeFavelier]

The next logical step is just to remove them from CircleCI I think

So you mean pip uninstall them then?

[GuillaumeFavelier]

until we change the default.

if mayavi is not available, pyvista will be selected automatically by get_3d_backend()

[GuillaumeFavelier]

For future reference then, there is a dependency to traits in 3d testing explaining the failure on Azure:

__________________________ test_link_brains[pyvista] __________________________
[gw0] win32 -- Python 3.7.6 c:\hostedtoolcache\windows\python\3.7.6\x64\python.exe
mne\utils\_testing.py:250: in dec
    with traits_test_context():
c:\hostedtoolcache\windows\python\3.7.6\x64\lib\contextlib.py:112: in __enter__
    return next(self.gen)
mne\utils\_testing.py:237: in traits_test_context
    from traits.api import push_exception_handler
E   ModuleNotFoundError: No module named 'traits'
_______________________ test_plot_evoked_field[pyvista] _______________________
[gw0] win32 -- Python 3.7.6 c:\hostedtoolcache\windows\python\3.7.6\x64\python.exe
mne\utils\_testing.py:250: in dec
    with traits_test_context():
c:\hostedtoolcache\windows\python\3.7.6\x64\lib\contextlib.py:112: in __enter__
    return next(self.gen)
mne\utils\_testing.py:237: in traits_test_context
    from traits.api import push_exception_handler
E   ModuleNotFoundError: No module named 'traits'
________________________ test_plot_alignment[pyvista] _________________________
[gw0] win32 -- Python 3.7.6 c:\hostedtoolcache\windows\python\3.7.6\x64\python.exe
mne\utils\_testing.py:250: in dec
    with traits_test_context():
c:\hostedtoolcache\windows\python\3.7.6\x64\lib\contextlib.py:112: in __enter__
    return next(self.gen)
mne\utils\_testing.py:237: in traits_test_context
    from traits.api import push_exception_handler
E   ModuleNotFoundError: No module named 'traits'
____________________ test_snapshot_brain_montage[pyvista] _____________________
    return next(self.gen)
mne\utils\_testing.py:237: in traits_test_context
    from traits.api import push_exception_handler
E   ModuleNotFoundError: No module named 'traits'

[larsoner]

if mayavi is not available, pyvista will be selected automatically by get_3d_backend()

I mean the default for users, not the CI machinery. Just by changing requirements.txt we don't change what users might already have installed, so many (probably most) users having mayavi installed already would see no change. "Changing the default" means (among other things) changing the order in which we try backends in our code, putting PyVista first.

[larsoner]

So you mean pip uninstall them then?

Yes exactly what I'm thinking

For future reference then, there is a dependency to traits in 3d testing explaining the failure on Azure:

It might make sense to actually have one of the runs not have mayavi at all, for reasons like this. Feel free to make that change. But like CircleCI, just pip uninstall instead of modifying requirements/environment until we are ready to fully jump to preferring/defaulting to pyvista

[GuillaumeFavelier]

I understand.

It might make sense to actually have one of the runs not have mayavi at all, for reasons like this. Feel free to make that change.

I think I will add that to #7162 because there are other considerations like which CI (Travis, Azure) and what job

[larsoner]

Actually it looks like one of the CI runs already has a pip uninstall -yq pysurfer mayavi

[larsoner]

https://github.com/mne-tools/mne-python/blob/master/azure-pipelines.yml#L129

[GuillaumeFavelier]

All good then

[GuillaumeFavelier]

I'll revert and modify Circle to to the same

[GuillaumeFavelier]

That would explain why we did not run into the traits dependency issue. We install mayavi with all its dependencies then only pip uninstall it.

[GuillaumeFavelier]

It's an item of #7162

[GuillaumeFavelier]

The following needs to be modified somehow:

LIBGL_DEBUG=verbose python -c "from mayavi import mlab; import matplotlib.pyplot as plt; mlab.figure(); plt.figure()"

[larsoner]

The from mayavi import mlab is just there to cause OpenGL to actually be accessed (in some way). If there is some equivalent for PyVista, feel free to add it. Maybe it's just:

import pyvista; pyvista.BackgroundPlotter(show=True)

you'll know if it works if CircleCI spews the same debug lines as it does on master. It's probably worth running CircleCI in SSH mode to figure out what works here. It's difficult to test locally because the LIBGL_DEBUG=verbose is only relevant if you're using MESA

[GuillaumeFavelier]

It seems that using:

import pyvista; pyvista.BackgroundPlotter(show=True)

works but it looks like the log is duplicated compared to master.

master	PR
https://app.circleci.com/pipelines/github/mne-tools/mne-python/1456/workflows/bcea3d26-286d-482a-b84f-cb75a948813d/jobs/19872/parallel-runs/0/steps/0-112	https://app.circleci.com/pipelines/github/mne-tools/mne-python/1462/workflows/c560824e-e3a1-49fb-b4d9-3589154fa720/jobs/19880/parallel-runs/0/steps/0-112

I will try next with just:

import pyvista; pyvista.Plotter()

[larsoner]

I recommend using circleci in SSH mode, you can iterate and test much faster that way. You get in, source $BASH_ENV, and should be good to go

[larsoner]

Actually the double logging might be due to the use of QGLWidget, IIRC this actually causes two contexts to be created even though only one is used. I bet it will be fixed in my PyVista PR. Might not be worth worrying about too much here even if the logging is doubled, it's really just there for debugging when we need to

[GuillaumeFavelier]

Might not be worth worrying about too much here even if the logging is doubled, it's really just there for debugging when we need to

Okay then. Should I start a [circle full]?

[larsoner]

Yep, let's see if it succeeds!

[agramfort]

do you need feed back here?

…

[GuillaumeFavelier]

I was waiting for the CIs but yes, this is basically ready for reviews @agramfort

[agramfort]

I see no objection to merge if CIs are green

[codecov]

Codecov Report

Merging #7702 into master will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #7702   +/-   ##
=======================================
  Coverage   90.24%   90.24%           
=======================================
  Files         455      455           
  Lines       84666    84666           
  Branches    13414    13414           
=======================================
+ Hits        76405    76406    +1     
  Misses       5393     5393           
+ Partials     2868     2867    -1     

[GuillaumeFavelier]

Something went wrong with CircleCI, the front page is completely broken on my browser:

https://19884-1301584-gh.circle-artifacts.com/0/dev/index.html

[larsoner]

That is happening on all PRs, I think it's some server configuration bug introduced by the CircleCI folks but haven't looked into if there is an open issue about it anywhere. If you look at the Chrome properties you'll see that none of the external JS libraries load anymore, including Boostrap. But things are fine on GitHub pages once they're deployed there.

[larsoner]

Upstream report filed

https://discuss.circleci.com/t/circleci-403-forbidden-in-html-artifacts/35791

[larsoner]

Thanks @GuillaumeFavelier !