Shaka DASH output doesn't pass conformance test with DASH validator

[ifij775]

System info

Operating System: Windows
Shaka Packager Version: 2.4.2

Issue and steps to reproduce the problem

1. Record mov video on my iphone.
2. Convert mov to mp4 (See in attached zip file)
   ffmpeg -i firepit.mov -c copy firepit.mp4
3. Run shaka packager
4. Upload to website
5. Submit link to https://conformance.dashif.org/

Packager Command:
firepit.zip

.\packager-win.exe in=firepit.mp4,stream=audio,output=dash_audio.mp4 in=firepit.mp4,stream=video,output=dash_video.mp4 --mpd_output dash.mpd
[0423/115916:INFO:demuxer.cc(88)] Demuxer::Run() on file 'firepit.mp4'.
[0423/115916:INFO:demuxer.cc(160)] Initialize Demuxer for file 'firepit.mp4'.
[0423/115916:INFO:single_segment_segmenter.cc(107)] Update media header (moov) and rewrite the file to 'dash_video.mp4'.
[0423/115916:INFO:mp4_muxer.cc(177)] MP4 file 'dash_video.mp4' finalized.
[0423/115916:INFO:single_segment_segmenter.cc(107)] Update media header (moov) and rewrite the file to 'dash_audio.mp4'.
[0423/115917:INFO:mp4_muxer.cc(177)] MP4 file 'dash_audio.mp4' finalized.
Packaging completed successfully.

What is the expected result?

Valid DASH output

What happens instead?

Fails dash conformance at https://conformance.dashif.org/
See warnings and errors below for video log and audio log.

VIDEO LOG:
***Legend: Errors, Warnings, Information ***

Information:
ImageDescription name must be '0' not '10'
According to DASH-IF IOP Section 3.2.8 @Bandwidth of the Representation (17268150 bps) is set too high given the @minimumBufferTime (2 s), the minimum @Bandwidth value required to conform is 12970380 bps.

Warning: major brand ('isom') not also found in list of compatible brands
WARNING: In moov-1:meta-1 - unknown meta atom 'ID32'

Error: ftyp-1
The brand 'isom' can only be a compatible, not major, brand

AUDIO LOG:
***Legend: Errors, Warnings, Information ***
Information:
SoundDescription numChannels must be 2 not 1
Validate_ES_Descriptor: ES_ID should be 0 not 1 in media tracks
According to DASH-IF IOP Section 3.2.8 @Bandwidth of the Representation (81270 bps) is set too high given the @minimumBufferTime (2 s), the minimum @Bandwidth value required to conform is 60150 bps.

Warning: major brand ('isom') not also found in list of compatible brands
WARNING: In moov-1:trak-1:mdia-1:minf-1:stbl-1 - unknown sample table atom 'sgpd'
WARNING: In moov-1:meta-1 - unknown meta atom 'ID32'

error: ftyp-1
The brand 'isom' can only be a compatible, not major, brand
error (256 repetition\s) ### grouping_type roll in sbgp is not found for any sgpd in moof number 1

[krzemienski]

I also am seeing similar on dashif conformance:

As of 2.4.2 release Im investigating playback issues relative to hevc based adapt sets, the avc based adapt sets play fine on all major ott players (roku exo). How ever Roku player does not like the hevc packaged content and wont start playback. Both the avc and the hevc source mp4s are encrypted and packaged with the below

def package_dash(input_dir, output_dir):
    trace = 'package_dash'

    pkg_commands = []
    packaging_cmd_to_execute = 'packager '
    seperator = ' '
    video_audio_hevc_avc_transcode_targets = glob.glob(f'{input_dir}/*.mp4')
    sidecar_targets = glob.glob(f'{input_dir}/*.vtt')

    packager_video_inputs_outputs_hevc_targets = []
    mpd_generator_paths_hevc = []
    packager_video_inputs_outputs_avc_targets = []
    mpd_generator_paths_avc = []
    packager_audio_inputs_outputs_targets = []
    mpd_generator_paths_audio = []
    packager_subtitle_inputs_outputs_webvtt_targets = []
    mpd_generator_paths_subtitle = []

    for mp4_file_name in video_audio_hevc_avc_transcode_targets:
        logger.info(f"{trace}: mp4 {mp4_file_name}")
        if ('hev1' in mp4_file_name or 'h265' in mp4_file_name) and '.vtt' not in mp4_file_name:
            mpd_generator_paths_hevc.append(f"{output_dir}/{Path(mp4_file_name).stem}"+"_cenc_protected.mp4.media_info")
            packager_video_inputs_outputs_hevc_targets.append(format_packager_in_target_audio_video(mp4_file_name, 'video', output_dir, 'MEDIA'))
        elif ('avc1' in mp4_file_name or 'h264' in mp4_file_name) and'.vtt' not in mp4_file_name:
            mpd_generator_paths_avc.append(f"{output_dir}/{Path(mp4_file_name).stem}"+"_cenc_protected.mp4.media_info")
            packager_video_inputs_outputs_avc_targets.append(format_packager_in_target_audio_video(mp4_file_name, 'video', output_dir, 'MEDIA'))
        elif 'audio' in mp4_file_name and '.vtt' not in mp4_file_name: 
            mpd_generator_paths_audio.append(f"{output_dir}/{Path(mp4_file_name).stem}"+"_cenc_protected.mp4.media_info")
            packager_audio_inputs_outputs_targets.append(format_packager_in_target_audio_video(mp4_file_name, 'audio', output_dir, 'MEDIA'))
    for sidecar in sidecar_targets:
        ##WHEN I GET TO THE POINT WE ACTUALLY KNOW THE LANGUAGE VIA PROVIDER OR ANALYZING PROPERLY WE JUST ADD ANOTHER PARAM
        ##FOR NOW I WILL JUST ASSUME AND DEFAULT ENGLISH
        mpd_generator_paths_subtitle.append(f"{output_dir}/{Path(sidecar).stem}"+".vtt.media_info")
        packager_subtitle_inputs_outputs_webvtt_targets.append(format_packager_in_target_text(sidecar, 'text', output_dir))
    
    packager_audio_videoh264_videoh265_inputs_outputs = seperator.join(packager_video_inputs_outputs_avc_targets) + " " + seperator.join(packager_video_inputs_outputs_hevc_targets) + " " + seperator.join(packager_audio_inputs_outputs_targets) + " " + seperator.join(packager_subtitle_inputs_outputs_webvtt_targets)

    ##APPEND ALL INPUTS TO THE PACKAGING COMMAND 
    packaging_cmd_input_options = packaging_cmd_to_execute + packager_audio_videoh264_videoh265_inputs_outputs

    ##LAST BUT NOT LEAST WE ADD IN THE ENCRYPTION KEYS TO APPLY DRM TO THE RESULTING PACKAING
    enable_media_info_output = '--output_media_info'
    flag_raw_encryption = '--enable_raw_key_encryption'
    key_and_keyid = f"--keys label={DRM_LABEL_DEFAULT}:key_id={KEY_ID}:key={KEY}"
    protections = '--protection_systems Widevine,PlayReady'
    mpd_out = f"--mpd_output {output_dir}/master-avc-hevc.mpd"

    packaging_cmd_to_execute = f"{packaging_cmd_input_options} {enable_media_info_output} {flag_raw_encryption} {key_and_keyid} {protections} {mpd_out}"
    pkg_commands.append(packaging_cmd_to_execute)


    #### DUE TO THE NEED FOR DIFFERENT MANIFEST FOR ALL TYPES OF PLAYER WE WILL GO AHEAD AND REFERENCE THE SAME ENCRYPTED MEDIA BUT 
    #### A FEW DIFFERENT VARIANTS ON THE PLAYLISTS
    
    manifest_seperator = ","
    infiles_hevc = manifest_seperator.join(mpd_generator_paths_hevc)
    infiles_avc = manifest_seperator.join(mpd_generator_paths_avc)
    infiles_audio = manifest_seperator.join(mpd_generator_paths_audio)
    infiles_text = manifest_seperator.join(mpd_generator_paths_subtitle)

    build_manifest_h264 = f"/transcoder_packager/utils/mpd_generator --input {infiles_avc},{infiles_audio},{infiles_text} --output {output_dir}/master.mpd"
    pkg_commands.append(build_manifest_h264)

    build_manifest_h265 = f"/transcoder_packager/utils/mpd_generator --input {infiles_hevc},{infiles_audio},{infiles_text} --output {output_dir}/master-hevc.mpd"
    pkg_commands.append(build_manifest_h265)

    for pkg_cmd in pkg_commands:
        try:
            ##EXECUTE THE PACKAGING AND ENCRYPTION COMMAND
            logger.info(f"{trace}: package_cmd to be executed {pkg_cmd}")
            with subprocess.Popen(pkg_cmd, env=my_env, shell=True, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True) as p:
                for line in p.stdout:
                    logger.info(f"{trace}: {line}", end='')
        except (OSError) as exception:
            logger.error('Exception occured: ' + str(exception))
            logger.error(f'{trace} failed')
            return False
    
     # no exception was raised
    logger.info(f'{trace} - all packaging finished')
    return True

Here are the produced manifests from the above.
shaka-2.4.2-avc-hevc-multi-codec-and-standalone-masters.zip

This might have to do with the fact that original content prior to being passed to packager command is or at one point was hevc mp4. I'm going to try some previous releases and see if I can get any to be liked by roku and also confirm that hevc in exo also doesn't work.

to note both avc and the hevc adapt sets even the ones that DO play fine with roku have the same errors in dashif conformance like so -

HEVC based adapt set error

***Legend: Errors, Warnings, Information ***

(2 repetition\s) ImageDescription name must be '0' not '11'

Warning: major brand ('isom') not also found in list of compatible brands
(2 repetition\s) Warning: moov-1:trak-1:mdia-1:minf-1:stbl-1:stsd-1 - unknown atom found "hvcC": video sample descriptions would not normally contain this
WARNING: In moov-1:meta-1 - unknown meta atom 'ID32'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'saiz'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'saio'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'senc'

### error: ftyp-1
### The brand 'isom' can only be a compatible, not major, brand
### error: moov-1:trak-1:mdia-1:minf-1:stbl-1:stsd-1
### MPEG-4 only allows 1 sample description
error### For an encrypted content, ContentProtection Descriptor shall always be present and DASH264 profile shall also be presentAccording to DASH-IF IOP Section 3.2.8 @bandwidth of the Representation (1714050 bps) is set too high given the @minimumBufferTime (2 s), the minimum @bandwidth value required to conform is 1621875 bps

AVC based adapt set error

***Legend: Errors, Warnings, Information ***

(2 repetition\s) Validate_ES_Descriptor: ES_ID should be 0 not 1 in media tracks

Warning: major brand ('isom') not also found in list of compatible brands
WARNING: In moov-1:meta-1 - unknown meta atom 'ID32'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'saiz'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'saio'
(514 repetition\s) WARNING: In moof-2:traf-1 - unknown traf atom 'senc'

### error: ftyp-1
### The brand 'isom' can only be a compatible, not major, brand
### error: moov-1:trak-1:mdia-1:minf-1:stbl-1:stsd-1
### MPEG-4 only allows 1 sample description
error### For an encrypted content, ContentProtection Descriptor shall always be present and DASH264 profile shall also be present According to DASH-IF IOP Section 3.2.8 @bandwidth of the Representation (134000 bps) is set too high given the @minimumBufferTime (2 s), the minimum @bandwidth value required to conform is 132658 bps.

Obviously only the hvcc atom delta can be noticed, does shaka not support hevc packaging?

I could have sworn in the past i've had no issue with it, obviously you are promoting vp9 and i know its not in the docs but maybe i just made an assumption? The dash that is generated though for the hevc based inputs is spec compliant so maybe that's why i never really second checked the support for hevc and shaka.

Sorry if i hijacked this issue, @kqyang let me know if you think this should be moved to its own issue. (also that python snippet is what i was talking about and will post in our other issue regarding mpd_generator so folks maybe can use it as a starting point)

[kqyang]

@krzemienski Yes, please file a separate issue instead. It is related but your issue is likely not caused by the incompatibilities.