[HOLD for Payment 2024-05-15][$500] mWeb - Chat - Infinite loading on video preview before sending video

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Version Number: 1.4.57-2
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4456414
Email or phone of affected tester (no customers): vdargentotest+mweb032724@gmail.com
Logs: https://stackoverflow.com/c/expensify/questions/4856
Issue reported by: Applause - Internal Team

Action Performed:

Pre-requisite: the user must be logged in

1. Go to any chat
2. Tap on "+" button
3. Select "Add attachment" and any other option
4. Select a video

Expected Result:

A video preview should be displayed before sending the video

Actual Result:

The video preview page keeps loading infinitely

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

Bug6428802_1711551855450.Gvyy1795_1_.mp4

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~015c647e39b68ccf23
• Upwork Job ID: 1773758325269905408
• Last Price Increase: 2024-05-01
• Automatic offers:
• paultsimura | Reviewer | 0
• ikevin127 | Contributor | 0
• ishpaul777 | Contributor | 0

[melvin-bot]

Triggered auto assignment to @alexpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[lanitochka17]

@alexpensify FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

[lanitochka17]

We think that this bug might be related to vip-vsp

[alexpensify]

Tested with the website and confirmed that it does load. Now, going to try in an iOS web browser.

[alexpensify]

I tested and confirmed this issue is only on iOS: mWeb Safari.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~015c647e39b68ccf23

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @paultsimura (External)

[alexpensify]

Waiting for proposals here

[alexpensify]

No update, we need proposals here.

[cleverjam]

I have spent some time looking into this but can't reproduce this in dev server, this is making it harder to debug.

iOS: 17.2

Simulator.Screen.Recording.-.iPhone.15.Pro.Max.-.2024-04-01.at.18.42.43.mp4

Edit: When I do manage to reproduce, only in staging + my actual physical device, I get the following error Failed to load resource: The operation couldn’t be completed. (WebKitBlobResource error 1.). There are some existing threads you can find by searching google on similar issues, but until I can reproduce consistently on my dev environment I can't confidently test a fix so any information on the issue is unusable to me at the moment. I am posting this hoping it will help someone else.

[paultsimura]

@cleverjam thanks for the investigation – indeed, it's only reproducible on physical devices. Just a side note: you can test your local dev env from a physical device as well, not only staging/production. Or do you only want to work with a simulator?

[cleverjam]

@paultsimura I have not been able to access local dev server from my physical device, it seems the dev server binds to 127.0.0.1 only? I may have missed the instructions to make this work (I am new here), but I followed this README, would you mind sharing instructions to access local dev from physical device?

[ikevin127]

Proposal

Please re-state the problem that we are trying to solve in this issue

The video preview keeps loading infinitely on iOS: mWeb.

What is the root cause of that problem?

Note

This can only be reproduced on real iOS devices, and it happens on all browser (Safari, Chrome, etc) since all iOS browsers use the WebKit engine.

The root cause comes from this code line:

App/src/components/VideoPlayer/BaseVideoPlayer.js

Lines 64 to 65 in 35b4d1b

	// we add "#t=0.001" at the end of the URL to skip first milisecond of the video and always be able to show proper video preview when video is paused at the beginning
	const [sourceURL] = useState(VideoUtils.addSkipTimeTagToURL(url.includes('blob:') || url.includes('file:///') ? url : addEncryptedAuthTokenToURL(url), 0.001));

The problem is that we're appending #t=0.001 to the video blob URL in order to get a visible preview of the video before user interacts with it, we did this to we avoin seeing nothing (transparent) preview.

Appending #t=0.001 to the blob causes the following conosle log error:
Failed to load resource: blob:https://...%23t=0.001
The operation couldn’t be completed. (WebKitBlobResource error 1.)
this causes the infinite loading. Removing the time skip addition solves the console log error and the infinite loading.

This is most likely a WebKit browser engine limitation which does not allow us to skip the first milisecond of the video programatically via #t= (time fragment), without user action like actually toggling play / pause of the video.

What changes do you think we should make in order to solve the problem?

Based on #39078 (comment) findings related strictly to iOS: mWeb (WebKit-based browser engine) and CME decision from #39078 (comment) -> the updated solution is:

1. Create a new Browser library function called isMobileWebKit which will return true for all iOS: mWeb browsers (including simulator) that use WebKit-based browser engine (Safari, Chrome, Firefox, etc).
2. Use newly added Browser.isMobileWebKit() to perform a check at the top of the addSkipTimeTagToURL function and return url; such that we don't add the #t= (time fragment) tag.

Once this is done, we won't get the blob error and the infinite loading spinner anymore. Given the known browser-limitation, the downside with this is that while it would fix the infinite loading issue / console log error, it will still leave us with a transparent preview of the video until the user will interact with it (see iOS: mWeb video below).

All other platforms work as expected: no errors and the preview showing correctly using the existing #t= (time fragment) implementation.

Videos

iOS: mWeb (before / after)

Before	After
before.MP4	ios-mweb.mov

[ikevin127]

would you mind sharing instructions to access local dev from physical device?

@cleverjam Try ngrok, once you instelled it and have the Expensify web app running locally, run the following command in a terminal: ngrok http https://127.0.0.1:8082 --host-header="127.0.0.1:8082"

This will create a tunnel to your web app running locally (given it's running on port 8082) and ngrok will give you a link of form: https://5740-2a02-2f0c-5802-a600-fd16-b4f4-e331-cc4d.ngrok-free.app which you can email yourself and navigate to it on your physical iOS device.

Once you do that you can debug / monitor your iOS device console logs from Safari (desktop) if the devices are connected via USB.

[cleverjam]

thanks for sharing @ikevin127 - very helpful!

[alexpensify]

@paultsimura when you get a chance, can you review the recent proposals? Thanks!

[paultsimura]

@ikevin127 thanks for your proposal. Unfortunately, the platform-specific solutions are highly discouraged.
Do you think using the playVideo().then(() => pauseVideo()) hack might be a solution for all platforms instead of using the timecode?

Also, could you please share a testing branch of your current solution? 🙇

[paultsimura]

Asked the author of the original PR as well: #36832 (comment)

[ikevin127]

Unfortunately, the platform-specific solutions are highly discouraged.

@paultsimura Well, this is a platform-specific issue - but I understand 🙂

Do you think using the playVideo().then(() => pauseVideo()) hack might be a solution for all platforms instead of using the timecode?

Yes, I think so since we also do something similar here (reversed pause / play).

So if we want to avoid the main solution point (1.) - using platform-specific code then we could simply revert the #t= fragment logic implemented by original PR here and only use the playVideo().then(() => pauseVideo()) to achieve the same thing (more or less) - which works universally on all platforms.

Here's the test branch including the above-mentioned logic -> see the code diff here.

Note

I noticed that on Android:Native Emulator sometimes the preview gets stuck at a full green screen. Not sure this is happening on real Android devices.

Test branch results:

Android: Native

android.mov

Android: mWeb Chrome

android-mweb.mov

iOS: Native

ios.MP4

iOS: mWeb Safari

ios-mweb.MP4

MacOS: Chrome / Safari

web.mov

Anyways if there's any issues with this implementation on any other platforms I think the best solution around that is to use the proposed isIOSWebkitBrowser() check so we don't affect the original PR logic which works well on all other platforms, if not necessary.

Asked the author of the original PR as well

I'm curious to hear @Skalakid's take as well before moving forward.

[melvin-bot]

@amyevans @alexpensify @ikevin127 @ishpaul777 this issue is now 4 weeks old, please consider:

• Finding a contributor to fix the bug
• Closing the issue if BZ has been unable to add the issue to a VIP or Wave project
• If you have any questions, don't hesitate to start a discussion in #expensify-open-source

Thanks!

[ikevin127]

@ishpaul777 Indeed, after extensive testing I can confirm too that none of my proposed solutions fixes the issue, specifically only on physical iOS: mWeb devices where all the browsers use WebKit engine by design.

This must be due to the browser limitation / privacy concerns - where a video cannot be played / seeked without actual user interaction (button click or press), hence not allowing us to extract a frame to use for preview using the above suggested methods.

I have a lead on how to possibly tackle this issue, I just have to test it first then I'll write-up a formal proposal, again.

[ikevin127]

I have a lead on how to possibly tackle this issue

Yeah...it's not possible (IMO) to have this issue fixed on iOS: mWeb on real physical devices (all WebKit based browsers) from the FE side. Why do I say this ?

The lead mentioned above was that I added an external https://somedomain.com/video.mp4 form video directly to the BaseVideoPlayer sourceURL to which the #t=0.001 (time fragment) was appended and this worked when the attachment modal was opened with the hardcoded external video.

The issue is that we cannot pass local files to the video player dirrectly for security reasons, unless we're transforming them in a URL blob via URL.createObjectURL with this logic. Problem is that on mWeb WebKit based browsers, appending #t=0.001 (time fragment) to a blob does not work, throwing the (WebKitBlobResource error 1.) error -> resulting in infinite loading spinner.

Given this, we most likely won't want to send local files to BE simply to generate a preview so this leaves us with the following possible solution that will do the following only on mWeb WebKit based browsers:

• remove the #t=0.001 (time fragment) addition in order to prevent the error and have the user blocked on the infinite loading spinner

This still won't show a video preview since it's literally impossible (IMO) since on mWeb WebKit based browsers we're not allowed to set #t=0.001 (time fragment) and we're also not allowed to play the video programatically in order to extract a frame -> leaving us without a video preview, but at least the user is not blocked from attaching the video, playing it and uploading it.

This is how it will look on mWeb WebKit based browsers:

iOS: mWeb Safari

RPReplay_Final1714183179.MP4

Please let me know if this is an acceptable solution in your opinion for this specific edge case in order to at least unblock the user. If so, I'd update my proposal to remove the #t=0.001 (time fragment) specifically for mWeb WebKit based browsers.

@amyevans @ishpaul777

[ishpaul777]

@amyevans What's your suggestion, should we wait for ideal solution that fixes the thumbnail issue or just move without showing preview to prevent infinite loading for now (its def. not ideal but unblocks the user)

[amyevans]

Let's just move forward with not showing a preview in mWeb. It's not great, but it's better than an infinite loader, and given this is a relatively niche flow I'm okay with accepting the limitation

[ishpaul777]

Cool! lets move this forward then, @ikevin127 Can you please update your proposal

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[ikevin127]

Cool! lets move this forward then, @ikevin127 Can you please update your proposal

Updated proposal

• updated solution according latest findings from #39078 (comment) and CME's decision from #39078 (comment)

Note

With the updated solution -> the behaviour will now align on both iOS: mWeb physical devices and Simulator as in both will have transparent preview even though the #t=1 (time fragment) bug does not happen on iOS: mWeb Simulator devices.

cc @ishpaul777

[amyevans]

Thanks for updating the proposal! Let us know once the PR is updated as well

[ikevin127]

Thanks for updating the proposal! Let us know once the PR is updated as well

PR #39755 was updated according to proposal and is ready for review! 🎉

Note

Sorry, I thought I'm supposed to wait for @ishpaul777 's approval of the updated proposal before moving on to updating the PR.

cc @amyevans

[ishpaul777]

not overdue, PR is up for final review

[alexpensify]

Thank you for the update @ishpaul777!

[ikevin127]

⚠️ Automation failed -> this should be on [HOLD for Payment [2024-05-15]] according to today's production deploy from #39755 (comment).

cc @alexpensify

[ishpaul777]

not overdue, waiting for pay date

[alexpensify]

Ok, Upwork is up to date for tomorrow. I'm adding the note to remember why there are two C+ here: #39078 (comment)

Now, automation should kick in with the summaries tomorrow.

[alexpensify]

Payouts due: 2024-05-15

• Contributor: $500 @ikevin127
• Contributor+: $500 @ishpaul777
• Contributor+: $250 @paultsimura for partial review here

Upwork job is here.

We are treating this one as a High issue, which is why the payment is $500 and not $250.

[alexpensify]

Closing - everyone has been paid via Upwork and I closed the job there.