Getting gapless audio when concatenating a sequence of videos A, B, C on top of a single audio track

[brAzzi64]

Hi,

I'm trying to play a single audio track on top of a concatenation of video segments, as described originally in #6103. After the fix 0a89d0e recently linked on that issue, the approach suggested by @tonihei on Jul 1, 2019 is working really well :)

The only problem remaining is that despite making sure the "segment" duration is set to a multiple of the sample rate of the audio file (in microseconds), you can still hear a small audio gap during playback.

Is there a way to get rid of this?

Here's a patch with my current code, that applies on top of ExoPlayer's sample code (branch dev-v2):
https://gist.github.com/brAzzi64/3a0b37c0cc9e9f079af8041ce9959f92

Please let me know if you see anything off here?

Thanks!

[ojw28]

@tonihei - Any guidance?

[tonihei]

Sorry, haven't got around to check yet. I'll update the issue once I have something.

[tonihei]

I finally found the time to check what's happening with your example. Sorry for the delay!

It looks as if the "gap" you can hear is the player briefly going into BUFFERING state just after the transition. This is because the video timestamps don't align with keyframes in the video and the player needs to decode all frames from the previous keyframe before it can continue rendering from the requested start position. For reference, the key frames are at 0, 5_333_333 and 10_666_666 in your file. If you use any of those keyframe start positions for the video, everything is nice and smooth. Note that if you use enableInitialDiscontinuity=false, you must ensure that the first sample is a keyframe. Otherwise the decoder will see timestamps jumping backwards and some decoders may just break completely when this happens.

I also discovered that there is also a small bug in our fMP4 extractor that causes rounding errors for those clipping points (we round them to milliseconds and then back to microseconds). Not super bad, but I'll fix that too.

One final remark: You don't need to use the long ClippingMediaSource constructor for the audio tracks because they don't use the initial discontinuity by default (because audio samples are always "keyframes"). This parameter only makes a difference for video tracks.