[BUG] 3d points with opacity < 1 are invisible

[bloyl]

On (some?) linux virtual machines that use mesa_gl rendering, you can't visualize 3d points with any level of opacity. This affects mne coreg, plot_alignment etc.

You can recreate the issue by running https://mne.tools/dev/auto_examples/visualization/plot_eeg_on_scalp.html#sphx-glr-auto-examples-visualization-plot-eeg-on-scalp-py.

the figure below shows 2 plots. On the right is the default plot. and on the left is after I manually adjust the opacity using mayavi settings.

this is with export MESA_GL_VERSION_OVERRIDE=3.3 and isn't affected by MNE_3D_OPTION_ANTIALIAS.

This is most likely a vtk bug with old versions of LLVM but it would be good to figure out a minimum version of LLVM.

From looking at #7976 it seems one of our test systems is close to my system. I wonder if we could evaluate this there?

pyvista: 0.25.3 {pyvistaqt=0.1.1, OpenGL 3.3 (Core Profile) Mesa 18.3.6 via llvmpipe (LLVM 7.0, 256 bits)}

[bloyl]

I should note this is a problem for both mayavi and pyvista backends.

here is the output from the new sys_info:

Platform:      Linux-3.10.0-1127.el7.x86_64-x86_64-with-glibc2.10
Python:        3.8.3 (default, Jul  2 2020, 16:21:59)  [GCC 7.3.0]
Executable:    /mnt/isilon/meg_lab/Linux_software/anaconda3_lab/envs/mne-vtk-test/bin/python
CPU:           x86_64: 4 cores
Memory:        23.4 GB

mne:           0.21.dev0
numpy:         1.18.5 {blas=mkl_rt, lapack=mkl_rt}
scipy:         1.5.0
matplotlib:    3.2.2 {backend=Qt5Agg}

sklearn:       0.23.1
numba:         0.50.1
nibabel:       3.1.1
cupy:          Not found
pandas:        1.0.5
dipy:          1.1.1
mayavi:        4.7.2.dev0
pyvista:       0.25.3 {pyvistaqt=0.1.1, OpenGL 3.3 (Core Profile) Mesa 18.3.4 via llvmpipe (LLVM 7.0, 256 bits)}
vtk:           9.0.1
PyQt5:         5.15.0

[agramfort]

I cannot replicate on my mac but I guess it's expected

…

[larsoner]

This is almost certainly a MESA bug, but VTK might give us some way to force the rendering to work. @bloyl instead of making both opaque, can you check instead if setting the head surface opacity to 0.999 works?

[bloyl]

if I set the opacity of the surface < 1 it disappears.

[larsoner]

Any way you can upgrade your MESA version? You have 18.3.4 and 18.3.6 is what CircleCI uses, and it produces all of our rendered docs...

[bloyl]

That is just what I was hoping to hear!

[larsoner]

... and if you want to try something else quicker before that, try running the same xvfb line as CircleCI, then run something like DISPLAY=:99 python some_script_that_saves_a_screenshot.py, and see if that works. If that works but locally rendering doesn't, it indicates that MESA is not the problem, but rather your X display setup somehow does not use the same switches as the xvfb instance (+extension GLX +render), but I perhaps nothing at all would work in this case.

If you're already using software rendering with the same options as CircleCI, no point in trying that, though.

You can also try CircleCI's LIBGL_DEBUG line and see if the output is the same. But based on the rendering info from PyVista, it should be.

[larsoner]

From the 18.3.5 and 18.3.6 release notes I don't see anything immediately suspicious, but who knows

[larsoner]

@bloyl any luck here?

[bloyl]

Unfortunately no. I don't have admin on the system in question so can't really dig into it my self. I do have an open ticket with IS to look at it. But I can't imagine it's high on their priority list. Feel free to close if you want. I can always reopen if their are mne changes that would help.

…

On Mon, Aug 10, 2020, 12:21 PM Eric Larson ***@***.***> wrote: @bloyl <https://github.com/bloyl> any luck here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7977 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKTXHNIQ63A3N2MPLGUFYDSAAM75ANCNFSM4OUPVSFA> .

[larsoner]

You could probably compile an osmesa/software rendered version locally in your home directory somewhere and then launch an Xvfb instance that uses it for OpenGL. For example:

$ LD_LIBRARY_PATH=/home/bloyl/wherever/it/compiled xvfb-run -f .auth mne coreg -s sample -d ~/mne_data/MNE-sample-data/subjects --trans ~/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw-trans.fif &
$ x11vnc -display :99 -localhost -auth .auth --simple-rendering &
$ vncviewer localhost

You can cut out the vncviewer business if you're doing it remotely. But these are the sorts of commands I'd use to test on a system that actually has a display. I think I've used this sort of thing before on my Ubuntu 20.04 system with NVIDIA graphics to spin up a Mesa/swrast-rendered mne coreg before. So I'm optimistic it might work for you.

So maybe you "just" need to compile a newer version of Mesa yourself locally...

[larsoner]

Locally (Ubuntu 20.04) I can run this:

$ wget https://archive.mesa3d.org//mesa-18.3.6.tar.xz
$ tar xf mesa-18.3.6.tar.xz
$ cd mesa-18.3.6
$ sudo apt-get install libx11-xcb-dev libxcb-glx0-dev libxcb-dri2-0-dev libxxf86vm-dev libxcb-xfixes0-dev libxcb-dri3-dev libxcb-present-dev 
$ ./configure --with-platforms=x11 --with-gallium-drivers=swrast --with-dri-drivers=swrast --disable-egl
$ make -j 4

And from there, use my NVIDIA drivers:

$ mne sys_info | grep pyvista | awk -F 'OpenGL' '{print $2}'
 4.5.0 NVIDIA 440.100 via GeForce GTX 650 Ti BOOST/PCIe/SSE2}

Or the builtin Ubuntu 20.04 mesa:

$ xvfb-run mne sys_info | grep pyvista | awk -F 'OpenGL' '{print $2}'
 3.3 (Core Profile) Mesa 20.0.8 via llvmpipe (LLVM 10.0.0, 256 bits)}

And the 18.3.6 one I just built almost works when I use an env with VTK 8.1.2:

$ MESA_GL_VERSION_OVERRIDE=3.3 \
  LIBGL_DEBUG=verbose \
  LIBGL_DRIVERS_PATH="${PWD}/lib" \
  LD_LIBRARY_PATH="$PWD/lib" \
  xvfb-run mne sys_info
libGL: OpenDriver: trying /home/larsoner/python/mesa-18.3.6/lib/tls/swrast_dri.so
libGL: OpenDriver: trying /home/larsoner/python/mesa-18.3.6/lib/swrast_dri.so
...
Fatal Python error: Segmentation fault

Current thread 0x00007f69e0114740 (most recent call first):
  File "/home/larsoner/anaconda3/lib/python3.7/site-packages/pyvista/plotting/plotting.py", line 702 in render
...

I can keep hacking at this, but before I put more effort into it, let me know if you can even get the necessary -dev libraries installed on your destination machine to try it.

For example, VTK9 fails with an informative error that GLSL1.5.0 must be supported, so building a newer 19.x or 20.x might work better...

[larsoner]

@olafhauk 18.3.4 is the same problematic MESA that @bloyl was using here. CircleCI uses 18.3.6 and does not have this rendering problem.

What Linux distribution are you on? I can look to see if there is some at least somewhat painless way for you to get a newer MESA to fix the rendering problems.

[olafhauk]

@olafhauk 18.3.4 is the same problematic MESA that @bloyl was using here. CircleCI uses 18.3.6 and does not have this rendering problem.

What Linux distribution are you on? I can look to see if there is some at least somewhat painless way for you to get a newer MESA to fix the rendering problems.

This is what I got from IT:

We’re running CentOS Linux release 7.8.2003 (Core)

As for the MESA drivers, the are fairly current for the distribution since they got patched when the OS was patched:
mesa-dri-drivers.x86_64 18.3.4-7.el7 @base
mesa-filesystem.x86_64 18.3.4-7.el7 @base
mesa-khr-devel.x86_64 18.3.4-7.el7 @base
mesa-libEGL.x86_64 18.3.4-7.el7 @base
mesa-libGL.x86_64 18.3.4-7.el7 @base
mesa-libGL-devel.x86_64 18.3.4-7.el7 @base
mesa-libGLES.x86_64 18.3.4-7.el7 @base
mesa-libGLU.x86_64 9.0.0-4.el7 @centos7.5-x86_64-0
mesa-libGLU-devel.x86_64 9.0.0-4.el7 @centos7.5-x86_64-0
mesa-libgbm.x86_64 18.3.4-7.el7 @base
mesa-libglapi.x86_64 18.3.4-7.el7 @base
mesa-libxatracker.x86_64 18.3.4-7.el7 @base

[bloyl]

I am also running some version of redhat/centos 7 so I suspect my setup is similar to @olafhauk 's .

I'm able to duplicate the steps from @larsoner (above) to build and try it using xvfb-run: getting similar results. Interestingly I also tried building mesa.18.3.4 using the same steps and received the same error.

I suspect that we'll need to try to reinstall vtk against mesa.18.3.6. I'm not too familar with using xvfb and xvfb-run but I might play with it some this weekend.

[larsoner]

I suspect that we'll need to try to reinstall vtk against mesa.18.3.6

I'm not sure what this means. VTK should just use whatever OpenGL libraries are available, and these should be set by using xvfb-run (or not), and what LD_LIBRARY_PATH and such are. Hence why I'm optimistic that some local compilation plus xvfb-run should work...

[bloyl]

you are probably correct.
It seems like the mesa compile isn't complete as it is not specfying any GSLS support

[bloyl@reslnvirtws05 mesa-18.3.6]$ LIBGL_DEBUG=verbose   LIBGL_DRIVERS_PATH="${PWD}/lib"   LD_LIBRARY_PATH="$PWD/lib" xvfb-run glxinfo | grep -i opengl
OpenGL vendor string: Mesa Project
OpenGL renderer string: Software Rasterizer
OpenGL core profile version string: 3.3 (Core Profile) Mesa 18.3.6
OpenGL core profile shading language version string: (null)
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.3 (Compatibility Profile) Mesa 18.3.6
OpenGL shading language version string: (null)
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 2.0 Mesa 18.3.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 1.0.16
OpenGL ES profile extensions:

Even if I attempt to force the issue it doesn't work.

(mne-vtk-test) [bloyl@reslnvirtws05 mesa-18.3.6]$ MESA_GL_VERSION_OVERRIDE=3.3 MESA_GLSL_VERSION_OVERRIDE=150  LIBGL_DEBUG=verbose   LIBGL_DRIVERS_PATH="${PWD}/lib"   LD_LIBRARY_PATH="$PWD/lib"   xvfb-run glxinfo | grep -i opengl
OpenGL vendor string: Mesa Project
OpenGL renderer string: Software Rasterizer
OpenGL core profile version string: 3.3 (Core Profile) Mesa 18.3.6
OpenGL core profile shading language version string: (null)
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.3 (Compatibility Profile) Mesa 18.3.6
OpenGL shading language version string: (null)
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 2.0 Mesa 18.3.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 1.0.16
OpenGL ES profile extensions:

I can't find any suggestions on configuring mesa, so i'm not sure what the best next step is.

[GuillaumeFavelier]

From #7977 (comment), I see that vtk 9.0.1 is used, you have the issue on a previous version of VTK as well? For example vtk 8.1.2 available on pypi?

[bloyl]

vtk 8.1.2 I don't think is available for the python currently in the environment (3.8.3). I could try to roll things back but at the end of the day I'm not sure if its worth it.

I'm currently leaning towards a docker approach, to just skip these types of issues. although If i could find the magic config flags for mesa that would solve it I'd probably do that too :)

[larsoner]

@bloyl it looks like the real problem is that we aren't setting the paths properly -- the llvmpipe driver is elsewhere. Check this out:

$ MESA_GL_VERSION_OVERRIDE=3.3 LIBGL_DRIVERS_PATH="${PWD}/lib/gallium"   LD_LIBRARY_PATH="$PWD/lib/gallium"   xvfb-run glxinfo | grep -i opengl
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: llvmpipe (LLVM 16.0, 256 bits)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 18.3.6
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.3 (Compatibility Profile) Mesa 18.3.6
OpenGL shading language version string: 3.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.0 Mesa 18.3.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00
OpenGL ES profile extensions:

And no compliants from PyVista, which does some VTK stuff that complains with the commands above:

$ MESA_GL_VERSION_OVERRIDE=3.3 LIBGL_DRIVERS_PATH="${PWD}/lib/gallium" LD_LIBRARY_PATH="$PWD/lib/gallium" xvfb-run python -c "import pyvista; print(pyvista.Report())"
--------------------------------------------------------------------------------
  Date: Thu Oct 08 12:42:46 2020 EDT

             Linux : OS
                 8 : CPU(s)
            x86_64 : Machine
             64bit : Architecture
           62.8 GB : RAM
            Python : Environment
      VMware, Inc. : GPU Vendor
llvmpipe (LLVM 16.0, 256 bits) : GPU Renderer
3.3 (Core Profile) Mesa 18.3.6 : GPU Version

  Python 3.8.5 (default, Jul 28 2020, 12:59:40)  [GCC 9.3.0]

            0.26.0 : pyvista
             9.0.1 : vtk
1.20.0.dev0+eec0aa2 : numpy
             2.9.0 : imageio
             1.4.3 : appdirs
             0.5.1 : scooby
             4.0.8 : meshio
3.3.1.post1016+gb4fd83cf7 : matplotlib
             0.2.0 : pyvistaqt
            5.15.0 : PyQt5
            7.15.0 : IPython
1.6.0.dev0+d38e9cb : scipy
            4.48.0 : tqdm
--------------------------------------------------------------------------------

@olafhauk you can try the lines above (adapted; apt-get will not work on CentOS) to build these drivers, then the line in this comment to see if PyVista will produce a report. If it does, you should be good to go by doing some variant of setting these env vars on your system.

[larsoner]

(I figured this out after trying a bunch of stuff that didn't work, then noticing in one of the links I hit in my searches https://docs.mesa3d.org/osmesa.html#building-osmesa it mentions that the Gallium-based driver is one directory deeper...)

[larsoner]

And actually it looks like the MESA_GL_VERSION_OVERRIDE=3.3 is actually not necessary in the above commands -- I get a 3.3 either way.

[larsoner]

... scratch that, it does help the non-core profile case, which might matter (?).

[olafhauk]

Hi - is there an update on this? I still couldn't get it to work in the development version, and am still talking to our IT guys to find out more about our graphics set up. Is it working for everyone else?

[larsoner]

@olafhauk something like this should work (you'll need to use the yum equivalents for apt-get):

$ wget https://archive.mesa3d.org//mesa-18.3.6.tar.xz
$ tar xf mesa-18.3.6.tar.xz
$ cd mesa-18.3.6
$ sudo apt-get install libx11-xcb-dev libxcb-glx0-dev libxcb-dri2-0-dev libxxf86vm-dev libxcb-xfixes0-dev libxcb-dri3-dev libxcb-present-dev 
$ ./configure --with-platforms=x11 --with-gallium-drivers=swrast --with-dri-drivers=swrast --disable-egl
$ make -j 4
$ export MESA_GL_VERSION_OVERRIDE=3.3
$ export LIBGL_DRIVERS_PATH="${PWD}/lib/gallium"
$ export LD_LIBRARY_PATH="$PWD/lib/gallium"
$ python -c "import pyvista; print(pyvista.Report())"

If this says it's using Mesa 18.3.6 instead of your builtin 18.3.4 then we're getting somewhere.

I made an Ubuntu 18.04 VM yesterday so that I could make VTK9 wheels. I actually already have a CentOS VM I could try the above procedure on. I'll give it a shot and if it works, I could probably bundle the lib/gallium directory and upload it for you and @bloyl to try out.

[larsoner]

@olafhauk @bloyl in my year-or-so-out-of-date CentOS 7 VM I did:

$ export MESA_GL_VERSION_OVERRIDE=3.3
$ glxinfo | grep "core profile version"
OpenGL core profile version string: 3.3 (Core Profile) Mesa 11.2.2

Then I did a variant of the compile commands above:

$ wget https://archive.mesa3d.org//mesa-18.3.6.tar.xz
$ tar xf mesa-18.3.6.tar.xz
$ cd mesa-18.3.6
$ sudo yum-builddep mesa
$ ./configure --with-platforms=x11 --with-gallium-drivers=swrast --with-dri-drivers=swrast --disable-egl
$ make -j 4
$ export LIBGL_DRIVERS_PATH="${PWD}/lib/gallium"
$ export LD_LIBRARY_PATH="$PWD/lib/gallium"
$ glxinfo | grep "core profile version"
OpenGL core profile version string: 3.3 (Core Profile) Mesa 18.3.6

I tar czfved the lib directory -- feel free to try it yourself. If it works, some suitable export statements in your .bashrc should allow you to make it permanent.

In order to actually try MNE commands I need to update the OS using sudo yum update which is taking forever. I'll post the before-after results once it finishes...

[larsoner]

Okay doing:

$ export MESA_GL_VERSION_OVERRIDE=3.3
$ sudo yum install python3 python3-devel  # gets 3.6.8
$ sudo yum install dbus-libs freetype libxcb* xcb-util-*  # updates+installs for PyQt5 5.15.1
$ python3 -m pip install --user --upgrade pip wheel
$ python3 -m pip install --user --only-binary ":all:" vtk numpy scipy matplotlib pyqt5
$ python3 -m pip install --user pyface traits traitsui mayavi mne pyvista pyvistaqt
$ mne sys_info
Platform:      Linux-3.10.0-1160.2.2.el7.x86_64-x86_64-with-centos-7.9.2009-Core
Python:        3.6.8 (default, Oct 13 2020, 16:18:22)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
Executable:    /usr/bin/python3
CPU:           x86_64: 4 cores
Memory:        Unavailable (requires "psutil" package)
mne:           0.21.2
numpy:         1.19.4 {blas=openblas, lapack=openblas}
scipy:         1.5.4
matplotlib:    3.3.3 {backend=Qt5Agg}

sklearn:       Not found
numba:         Not found
nibabel:       Not found
cupy:          Not found
pandas:        Not found
dipy:          Not found
mayavi:        4.7.2
pyvista:       0.27.2 {pyvistaqt=0.2.0, OpenGL 3.3 (Core Profile) Mesa 18.3.4 via llvmpipe (LLVM 7.0, 256 bits)}
vtk:           9.0.1
PyQt5:         5.15.1
$ mne coreg -d subjects -s sample --trans MEG/sample/sample_audvis_raw-trans.fif --fif MEG/sample/sample_audvis_raw.fif --simple-rendering

Replicated the translucency problem, there are no points shown:

Then doing the build instructions above were not quite enough. I adjusted them based on this site to install and use LLVM properly, and to include EGL and GLES (why not):

$ sudo yum install llvm-devel
$ ./configure --enable-driglx-direct --enable-gles1 --enable-gles2 --enable-glx-tls --with-egl-platforms='drm x11' --with-platforms=x11 --with-gallium-drivers=swrast --with-dri-drivers=swrast --enable-llvm
$ make -j 4
$ mv lib/gallium/* lib/ && rmdir lib/gallium
$ export LIBGL_DRIVERS_PATH="${PWD}/mesa-18.3.6/lib"  # !! NOTE: Cannot use ~, must use full path !!
$ export LD_LIBRARY_PATH="${PWD}/mesa-18.3.6/lib"
$ glxinfo | grep "OpenGL core profile version"
OpenGL core profile version string: 3.3 (Core Profile) Mesa 18.3.6
$ mne sys_info
...
pyvista:       0.27.2 {pyvistaqt=0.2.0, OpenGL 3.3 (Core Profile) Mesa 18.3.6 via llvmpipe (LLVM 3.4, 256 bits)}
...
$ mne coreg -d subjects -s sample --trans MEG/sample/sample_audvis_raw-trans.fif --fif MEG/sample/sample_audvis_raw.fif --simple-rendering

seemed to fix things:

I've updated the link to the archive of the lib/ directory that you should export as your LIBGL_DRIVERS_PATH and LD_LIBRARY_PATH. Hopefully it's that simple for you @olafhauk @bloyl ... optimistically closing just in case :)

[drammock]

this info should probably make its way into the advanced install docs... I think there's already a section on troubleshooting 3Dviz, it should go there. Any volunteers?

[larsoner]

First let's wait to hear from @bloyl and @olafhauk if the libraries + export work for them. Then I can quickly move a variant of these instructions to the 3D viz troubleshooting

[LeilaNS]

Hi,
@larsoner
I can't use mne coreg. I can' see head shape.

I did most of update that you suggeted, the issue isn't fixed, now I got a warning
WARNING: Imported VTK version (9.0) does not match the one used
to build the TVTK classes (8.1). This may cause problems.
Please rebuild TVTK.

[agramfort]

it seems you updated vtk since you setup your MNE environment. I would recommend you create a fresh / new env following: https://mne.tools/dev/install/mne_python.html#installing-mne-python-and-its-dependencies let me know if I can help.

…

[LeilaNS]

But I did it yesterday!

[larsoner]

@LeilaNS what is your mne sys_info or python -c "import mne; mne.sys_info()"

[LeilaNS]

@larsoner
I reinstalled everything, now mne coreg works well, my problem was some file weren't update.
many thanks

[larsoner]

Closing as I think we have fixed bugs with 3D option setting, improved the docs on how to deal with at least some of these issues, and warn if an old version of MESA is used