WebXR VR Examples are broken since the v130 update on Oculus Quest 2

[dbradleyfl]

Describe the bug

A bunch of the three js examples for VR no longer work since v130 and we're seeing the same issue in our application when we upgrade to v130.

To Reproduce

Steps to reproduce the behavior:

1. It may require testing a layers enabled example first (or not)
2. Try the following examples:

• https://threejs.org/examples/?q=webxr#webxr_vr_ballshooter
• https://threejs.org/examples/?q=webxr#webxr_vr_cubes
• https://threejs.org/examples/?q=webxr#webxr_vr_panorama
• https://threejs.org/examples/?q=webxr#webxr_vr_panorama_depth
• https://threejs.org/examples/?q=webxr#webxr_vr_paint
• https://threejs.org/examples/?q=webxr#webxr_vr_rollercoaster
• https://threejs.org/examples/?q=webxr#webxr_vr_sculpt
• https://threejs.org/examples/?q=webxr#webxr_vr_video

Expected behavior

The demos should function as they did before, but currently they just show up totally black in the headset.

Platform:

• Device: Oculus Quest 2
• OS: v30.1
• Browser: Oculus Quest Browser
• Three.js version: 130

[dbradleyfl]

Using the webxr_vr_panorama as an example I found two things that "fix" the example.

1. It seems to be essential that the renderer has shadowMap true
   

2. It also seems to be essential that the scene have a light that casts shadows:
   

I don't think those new requirements are intentional.

[marcofugaro]

May be because of #22060 (comment)?

[Mugen87]

I can't reproduce on a Quest 2 with latest Oculus Browser (16). Is it possible that your browser is not up-to-date?

[cabanier]

These used to work on my quest. I suspect I broke something up while fixing the multiple render target problem.

[arpu]

@cabanier latest OculusBrowser update from 8h breaks vrland.io too, works bevore the Browser update yesterday

[cabanier]

@arpu , did you pick up a new three.js or is it just broken?

[arpu]

tested yesterday threejs 130 and all works, after todays browser update its just broken

[cabanier]

tested yesterday threejs 130 and all works, after todays browser update its just broken

If it breaks with a new browser and an old three.js, that sounds like another issue in the browser.
What happens if you disable WebXR Layers in chrome://flags? Is it still broken?

[arpu]

yes disable Webxr Layers in the chrome flags fixed the issue and i can enter webxr mode on vrland.io and threejs officiell samples

[arpu]

@cabanier tested your #22074, -> i can enter again with enabled webxr Layers but the quality is very poor looks like anti alias is not working with this ( webxr layer enabled/disabled)

[arpu]

with stock threejs 130 and disables Webxr Layers in OculusBrowser the result looks fine, hope this Helps

[cabanier]

Can you point me to a url or link to your github so I can see it?

[arpu]

only testing this local, not pushed to any repo

[cabanier]

only testing this local, not pushed to any repo

I'm worried about this regression so let me know if you have a way to create a test case.

[arpu]

yes: i updated https://vrland.io/lobby ( v0.22.1 right top corner) right now with your new webxr regression fix from #22074

result:
enabled webxr Layer -> webxr enter works no antialias
disable webxr Layer -> webx enter works, antialias works

[dbradleyfl]

@cabanier @mrdoob fix confirmed on my end. Thanks @cabanier ! I tested the examples on my Oculus Quest 2 with the latest version of the dev branch. Any chance this will go out as 130.1 in the near future?

[cabanier]

I investigated this and realized that we're not exposing the multisample webgl extension.
This means that content using layers can't have AA until we add that.

Should we pull layers support for now? Alternatively, we can disable layers if AA is requested but that seems confusing.

[cabanier]

Alternatively, we can use more of three.js' code and tell it to render to the textures using the multisampled render target.
@Mugen87 @mrdoob I think you are more familiar with this code. Can you tell me how I can do that?

[arpu]

i think in this Bug request we should concentrate to the "normal" https://threejs.org/docs/#api/en/renderers/WebGLRenderer antialias ( multisampler render target is not needed for this)

[cabanier]

i think in this Bug request we should concentrate to the "normal" https://threejs.org/docs/#api/en/renderers/WebGLRenderer antialias ( multisampler render target is not needed for this)

The "normal" antialias uses the browser's builtin support. Unfortunately this works only if the framebuffer is under the control of the browser. With layers, the fb is created by the author so it has to make the necessary calls to make sure that it's rendering to the textures with AA.
Renderbuffers allow this but they are awkward to use. However, it seems that three.js already has logic to enable them.

The right solution is to enable the GL extension that the browser is using under the hood.

[Mugen87]

Alternatively, we can use more of three.js' code and tell it to render to the textures using the multisampled render target.

Then we need to merge #22066 first. Otherwise it is not possible to resolve the multisampling to the default framebuffer.

[Mugen87]

Any chance this will go out as 130.1 in the near future?

I guess the severity of the issue justifies a new minor release.

[cabanier]

We'll need to fix the anti-aliasing as well.

I will explore getting support for EXT_multisampled_render_to_texture as it is a much simpler and efficient way to get antialiasing on mobile GPU's

[Mugen87]

The regression should be fixed now in prod.

[arpu]

hmm but still the antialiasing Problem with enabled Layers?

[Mugen87]

That's not part of the regression. Probably better to open a separate issue for this.

[arpu]

ok tested it right now and still a problem for latest threejs and oculusBrowser default enabled Layers ( tested with the webxr ball sample)
@cabanier this needs to be fixed with a new OculusBrowser version?

[cabanier]

@cabanier this needs to be fixed with a new OculusBrowser version?

It won't be fixed with a new Oculus browser version.
@Mugen87 how hard is it to apply multisampling to the layer after you fixed #22066? Does it have a performance impact?

[mrdoob]

@cabanier I'm still catching up with all this... How come we lost antialias?

[cabanier]

AA in regular WebXR is done by using FramebufferTexture2DMultisampleEXT under the hood.
This extension is not (yet) available in WebGL but it is possible to have to same behavior using the GL commands that three.js is using for multisampled output.

If this is proving too hard, we can keep support for layers but revert to drawing to the XRWebGLLayer object that was used previously.

[mrdoob]

Hmm... So before we were using XRWebGLLayer and now we're using XRWebGLBinding.
Is this needed in order to be able to use XRMediaBinding?

[cabanier]

no, you can mix them.
XRWebGLBinding is used to create the new layer styles and XRMediaBinding is used to create media layers.
XRWebGLLayer can just be created. We added XRProjectionLayer because it is more efficient

[Mugen87]

@Mugen87 how hard is it to apply multisampling to the layer after you fixed #22066? Does it have a performance impact?

This is something I have to find out. Meaning investigating the performance differences of default/built-in MSAA with the manual approach. BTW I have closed #22066 accidentally. I'll make a new PR as soon as the API is clear (see #22066 (comment)).

[cabanier]

@mrdoob is there a time constraint?
If it takes too long, I can switch back to the old style layer.

[mrdoob]

@cabanier Then I think it'll be best to revert the changes in WebXRManager until we have a clear path to support antialias.

[cabanier]

I have a patch in progress in the Oculus browser that exposes an extension to bring AA back. (see cabanier@9350b5a#diff-a27b9d3ba68457069cc8ff521208ae9d0cc6d9256f6f48ada6316fc2d732ba7eR227)

For now, I agree that we should revert.

[mrdoob]

I see... Would you like to do the PR that reverts that part or should I give it a go?

[cabanier]

Let me try fixing it using 'renderbufferStorageMultisample'. If that doesn't work, I'll revert it.

I'm also trying to fast-track the webgl extension.

[cabanier]

#22127 will work around the AA issue until I can get the extension approved.

There is a bug in the browser that stops webgl layers from blending with other layers. This will be fixed in the oculus browser in 2 weeks.