Any way to control the resolution of audio-position-changed?

[jagthedrummer]

I've registered for the audio-position-changed event which I'm using to keep my waveform player up to date. I've noticed that the player updates kind of jerk and stutter and it seems like that's due to the audio-position-changed event only firing about twice a second at most. Is there any way to get that event to fire more often?

[jkeen]

Hmmm, are you subscribed to that event on the stereo service or on the sound itself?

There's a 500ms poller on the stereo service that controls the .position property on a sound, but i don't think that should affect the timing of event firing? I could be wrong though. You could try overriding the ember-stereo service and setting pollInterval to something small to see if that makes a difference? I don't think it will, though…

Another thing worth trying for your situation is starting your own loop with a much smaller delay where you update the waveform if the sound is playing? The mozilla docs for timeupdate say that event firing frequency can vary, so maybe dialing something in custom using an ember-concurrency task and/or a requestAnimationFrame call might be the way?

[jagthedrummer]

I originally subscribed to that event on the sound, but I tried subscribing on the stereo service but that doesn't seem to make a difference.

I tried overriding pollInterval by making a service like this:

import Service from '@ember/service';
import stereo from 'ember-stereo/services/stereo';

export default class CustomStereoService extends stereo {
  pollInterval = 50;
}

And then I'm injecting it into my component like this:

@service('custom-stereo') stereo;

When using that service I can call play and it will start the sound playing, but the audio-position-changed event continues to fire at the same rate AND all of the helpers like sound-is-playing and sound-is-loaded stop working. Maybe I'm doing something wrong with the override? If not I'll investigate the other suggestions you made.

[jkeen]

Hmm, not sure why the helpers would quit working, but yeah that is what I was thinking.

I'd try the other suggestion then? From what I saw in some searches, it's not that the audio element doesn't have updated data, it just doesn't notify that it does at a really high frequency.

What kind of things are you updating on the waveform player using that audio-position-changed event?

[jagthedrummer]

@jkeen I'm just updating the cusor position as it travels across the waveform and it just kind of jumps in half second increments instead of smoothly flowing. I didn't really mind that so much at first but then I started working on displaying regions and highlighting comments associated with those regions and I discovered that region-in and region-out events (in wavesurfer) weren't firing for short regions, and I finally tracked that down to the infrequent audio-position-changed updates making it so that sometimes the cursor would just skip over a short region without entering and exiting it.

[jkeen]

Oh ok, interesting. I'd try making an ember concurrency task that loops and monitors the position manually on the currently playing sound (perhaps even using the private _currentPosition() method, or getting a handle on the actual .audioElement through the sound object and using the currentTime property on the audio element itself) to see if that gets you there.

[jagthedrummer]

Thanks for the tip on using the private _currentPosition() method, that one did the trick along with a looping task. With both of those the playback motion on the cursor is nice and smooth.

So now the relevant bits of the component look something like this:

  togglePlaySoundTask = task(async () => {
    await this.sound.togglePause();
    if(this.sound.isPlaying){
      this.pollForChanges.perform();
    }
  });

  pollForChanges = task(async () => {
    while(this.sound.isPlaying) {
      let position = this.sound._currentPosition();
      let timeInSeconds = position / 1000;
      await this.wavesurfer.setTime(timeInSeconds);
      await timeout(60);
    }
  });

I found that without the await timeout() line that playback hangs for some reason. Not sure why. It can be as small as 0 and it still works, but if the line is removed entirely it doesn't. 🤷

And if I use this.sound.position instead of this.count._currentPosition() the cursor is still jerky due to sound.position not being updated with the same frequency as whatever is going on under the hood in _currentPosition(). It just updates several times in a row at one timestamp (say 0.55s) before leaping to the next increment of roughly 500 ms (say 1.05).

[jkeen]

That task modifier syntax is nice—I realized my brain's default is still the decorator-style syntax like this:

@task
*pollForChanges() {
   while (this.sound.isPlaying) {
      let position = this.sound._currentPosition();
      let timeInSeconds = position / 1000;
      yield this.wavesurfer.setTime(timeInSeconds);
      yield timeout(60);
  }
}

but I guess I need to update my settings!

Glad that solved it, but sorry for the hassle! The timeout is definitely key or else it just locks the browser, and I think doing a timeout(0) is probably equivalent to a runloop#next in terms of timing? I think anything between 60-150ms would probably be acceptable visually to redraw that wavesurfer display.

It would be nice to solve this in ember-stereo, though. I can definitely improve the frequency this.sound.position gets updated and when I do that I wonder if manually firing that audio-position-changed event instead of relaying it from the audio element would resolve issues like this, because it certainly would be a nicer experience if you could just subscribe to that event and call update on the wavesurfer display.

[jkeen]

@jagthedrummer Just released 4.3.0 that should fix this issue! Subscribing to audio-position-changed should work as you'd expect now.

[jagthedrummer]

Awesome, @jkeen! I'll give it a try tomorrow. I'd already discovered that my implementation was less than ideal and performed pretty poorly if the timeout was too low. I was going to look at using animation frames to make sure that it wasn't firing too often, but it looks like you already have that in yours. Will give it a try and report back.

[jagthedrummer]

Just wanted to report back that the fix in 4.3.0 is working great for me. I went back to just registering for the audio-position-changed and then using the sound.position attribute and updates are happening way more often than they had been previously. At least once every 100ms or so. The animation of the cursor isn't quite as smooth as with my pollForChanges implementation, but I think that's a good thing because I think I was polling too often. CPU usage is definitely higher with the pollForChanges method, and every so often the cursor would stutter or jump backwards, which I assume was due to clobbering/contention for the cursor position since I wasn't requesting animation frames and was just YOLOing updates as they came.