Added support for Thumbnails in DASH

[gorkemg]

This PR enhances the DashManifest class to allow fetching Thumbnail Meta Data for a video position and with that allowing to display thumbnail images while seeking through a video, as discussed in issue #3752 by my colleague @suraneau.

This approach is similar to how dash.js handles thumbnails. Only the metadata (like the URL) for the thumbnail is provided by ExoPlayer via a new API. The tasks of loading, caching, crop/zoom and rendering of the (tile) images will be done by the external app/UI logic.

The demo app has been enhanced as well to use this new API and display thumbnail images above the seek bar. A few streams have been added for testing.

We're looking forward to your feedback.
Thank you!

Best regards,
Görkem

[tonihei]

Thanks for sharing the code in a PR!

I think the general direction doesn't yet fully align with the discussion in #3752 because it looks like your proposal is moving the entire loading and caching out of the player to let the app handle this separately. This is not ideal because it means the app has to replicate many parts of the logic that lives inside the player already (how to load data, how to cache it, find the appropriate next sample after a seek, handling adaptive switches between resolutions etc). As a first step, this may still be appropriate just to make sure an app can do this if wants to.

The app can access the DashManifest object directly by calling (DashManifest) player.getManifest(). So the goal should be to parse and represent the thumbnail segments inside the usual AdaptationSet / Representation / SegmentTemplate classes. The grid size of the thumbnail images can be set in the Format object via a new image-specific field (e.g. tileCountHorizontal, tileCountVertical). This corresponds to step 1 in #3752 (comment).
After this is done, the remaining code can live in each individual app for now until we get around to handle the other mentioned improvements.

[gorkemg]

Hi @tonihei,

thank you for your very quick response.

As my colleague @suraneau mentioned in the discussion in #3752 , we were aiming to implement the approach similar to dash.js by offering a thumbnail API to easily fetch thumbnail information for App developers - as requested by companies in the discussion and as typically needed for DASH playback.
As you mentioned, we also understand that it makes sense to reuse logic already available in the player.
However, just like you wrote, we think this is a great first step in that direction.
With this PR merged, App developers can already start implementing the thumbnail seek bar into their apps for DASH playback, even reusing the seek bar we provided in the ExoPlayer demo application.

I looked into your suggestion for adding two new properties, tileCountHorizontal and tileCountVertical, to Format, so that the DashManifestParser can parse that information and have it available in the DashManifest by simply using representation.format.tileCountHorizontal, since this is a critical detail missing currently in the DashManifest.

I fully agree and adjusted Format accordingly by adding these two new properties in my last commit.
Additionally, DashManifestParser will now parse the EssentialProperty tag inside an Image AdaptationSet and parse these two values into Format.
With this change, our new method getThumbnailDescriptions() can simply access this data without any additional parsing.

We're again looking forward to your feedback.
Thank you!

Best regards,
Görkem

[tonihei]

Thanks for the updates! Could you just keep the changes to DashManifest(Parser) and Format and remove the remaining changes from the PR? As far as I see they are not needed for the initial parsing proposal and can be added/implemented by apps as required.

[gorkemg]

Hi @tonihei,
I removed the files and also merged dev-v2 again to remove a merge conflict in Format.
Looking forward to your feedback.
Thanks!

Best regards,
Görkem

[tonihei]

Thanks for the update! I'll merge the changes internally, but without the ThumbnailDescription part (which can be done in app code if needed).