Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging fails on MPEG-TS with SAMPLE-AES #1316

Closed
chenda6 opened this issue Dec 8, 2023 · 3 comments · Fixed by #1323 or #1312
Closed

Packaging fails on MPEG-TS with SAMPLE-AES #1316

chenda6 opened this issue Dec 8, 2023 · 3 comments · Fixed by #1323 or #1312
Labels
status: archived Archived and locked; will not be updated

Comments

@chenda6
Copy link

chenda6 commented Dec 8, 2023

System info

macOS Monterey 12.6
(cmake) https://github.com/shaka-project/shaka-packager/tree/cmake-porting-history

Issue and steps to reproduce the problem

Packager Command:

packager \                                                                                
        in=input.mp4,stream=video,segment_template=\$Number\$.ts,drm_label=sd \
        --enable_raw_key_encryption \                                                     
        --keys label=sd:key_id=abba271e8bcf552bbd2e86a434a9a5d9:key=69eaa802a6763af979e8d1940fb88392:iv=22334455667788990011223344556677 \
        --protection_scheme cbcs \                                                        
        --protection_systems FairPlay \                                                   
        --hls_master_playlist_output h264_master.m3u8 \                                   
        --hls_key_uri skd://testAssetID  

What is the expected result?
MPEG-TS segments successfully encrypted and packaged.

What happens instead?
Packager terminates early with following stack trace:

E1208 10:04:23.204043       2 aes_encryptor.cc:145] Expecting output size of at least 32 bytes.
F1208 10:04:23.204063       2 encryption_handler.cc:400] Check failed: encryptor_->Crypt(source, source_size, dest, &dest_size) 
*** Check failure stack trace: ***
    @        0x1085cef60  absl::lts_20230125::log_internal::LogMessage::PrepareToDie()
    @        0x1085ce818  absl::lts_20230125::log_internal::LogMessage::SendToLog()
    @        0x1085cdb04  absl::lts_20230125::log_internal::LogMessage::Flush()
    @        0x1085cf42c  absl::lts_20230125::log_internal::LogMessageFatal::~LogMessageFatal()
    @        0x1085cf47c  absl::lts_20230125::log_internal::LogMessageFatal::~LogMessageFatal()
    @        0x107d41800  shaka::media::EncryptionHandler::EncryptBytes()
    @        0x107d40280  shaka::media::EncryptionHandler::ProcessMediaSample()
    @        0x107d3f148  shaka::media::EncryptionHandler::Process()
    @        0x108090244  shaka::media::MediaHandler::Dispatch()
    @        0x107c9fa28  shaka::media::MediaHandler::DispatchMediaSample()
    @        0x107c9ef50  shaka::media::ChunkingHandler::OnMediaSample()
    @        0x107c9e120  shaka::media::ChunkingHandler::Process()
    @        0x108090244  shaka::media::MediaHandler::Dispatch()
    @        0x107c9fa28  shaka::media::MediaHandler::DispatchMediaSample()
    @        0x107d5cfbc  shaka::media::Demuxer::PushMediaSample()
    @        0x107d5c510  shaka::media::Demuxer::NewMediaSampleEvent()
    @        0x107d69514  _ZNSt3__1L8__invokeIRMN5shaka5media7DemuxerEFbjNS_10shared_ptrINS2_11MediaSampleEEEERPS3_JjS6_EvEEDTcldsdescT0_fp0_fp_spscT1_fp1_EEOT_OSC_DpOSD_
    @        0x107d69428  std::__1::__apply_functor<>()
    @        0x107d693ac  std::__1::__bind<>::operator()<>()
    @        0x107d6932c  _ZNSt3__1L8__invokeIRNS_6__bindIMN5shaka5media7DemuxerEFbjNS_10shared_ptrINS3_11MediaSampleEEEEJPS4_RKNS_12placeholders4__phILi1EEERKNSC_ILi2EEEEEEJjS7_EEEDTclscT_fp_spscT0_fp0_EEOSL_DpOSM_
    @        0x107d692d0  std::__1::__invoke_void_return_wrapper<>::__call<>()
    @        0x107d69250  std::__1::__function::__alloc_func<>::operator()()
    @        0x107d67fc0  std::__1::__function::__func<>::operator()()
    @        0x107e867e0  std::__1::__function::__value_func<>::operator()()
    @        0x107e7e3d4  std::__1::function<>::operator()()
    @        0x107e78a78  shaka::media::mp4::MP4MediaParser::EnqueueSample()
    @        0x107e778d4  shaka::media::mp4::MP4MediaParser::Parse()
    @        0x107d5b0ac  shaka::media::Demuxer::Parse()
    @        0x107d59f08  shaka::media::Demuxer::Run()
    @        0x107bed964  shaka::media::Job::Run()
    @        0x107bf0a84  _ZNSt3__1L8__invokeIMN5shaka5media3JobEFRKNS1_6StatusEvEPS3_JEvEEDTcldsdescT0_fp0_fp_spscT1_fp1_EEOT_OSA_DpOSB_
    @        0x107bf09cc  std::__1::__thread_execute<>()
    @        0x107bf0128  std::__1::__thread_proxy<>()
    @        0x197a8826c  _pthread_start
    @        0x197a8308c  thread_start

NOTE:
Using packager version 2.6.1 works. This appears to be a regression related to https://github.com/shaka-project/shaka-packager/pull/1119/files

@synologic
Copy link

synologic commented Dec 13, 2023

Confirmed, I am experiencing the same situation this effectively breaks Fairplay encryption.
OS is Linux, packager version 3e71302ba4-release

@synologic
Copy link

At line 145, the check should be:
if (*ciphertext_size < required_ciphertext_size - AES_BLOCK_SIZE)

due to RequiredOutputSize function stating:
// mbedtls requires a buffer large enough for one extra block.

So required_ciphertext_size is 32 bytes due to mbedtls requirement whereas the actual ciphertext_size is 16 bytes

@cosmin
Copy link
Contributor

cosmin commented Dec 25, 2023

@chenda6 can you try this with the fix-integration-tests branch which contains the totality of the 5 most recent pull requests I opened. Specifically the fixes in #1323 should resolve this specific issue.

Brainiarc7 added a commit to Brainiarc7/shaka-packager that referenced this issue Jan 16, 2024
joeyparrish pushed a commit that referenced this issue Feb 8, 2024
The current mbedtls integration was not working for some modes. See for
example #1316 and also lots of failing integration tests.

For example in pattern encryptor it works on one block at a time so it
cannot assume it's going to always get a buffer with a padding for an
extra block.

From what I can tell when the padding mode is correctly set to
`MBEDTLS_PADDING_NONE` there is no extra block being written to or
required.

This passes all crypto unit tests and integration tests.

Closes #1316
joeyparrish pushed a commit to joeyparrish/shaka-packager that referenced this issue Mar 12, 2024
🤖 I have created a release *beep* *boop*
---


##
[4.0.0](v3.0.2...v4.0.0)
(2024-03-12)


### ⚠ BREAKING CHANGES

* Update all dependencies
* Drop Python 2 support in all scripts
* Replace glog with absl::log, tweak log output and flags
* Replace gyp build system with CMake

### Features

* Add input support for EBU Teletext in MPEG-TS
([shaka-project#1344](https://github.com/joeyparrish/shaka-packager/issues/1344))
([71c175d](71c175d))
* Add install target to build system
([3e71302](3e71302))
* Add PlayReady support in HLS.
([shaka-project#1011](https://github.com/joeyparrish/shaka-packager/issues/1011))
([96efc5a](96efc5a))
* add startwithSAP/subsegmentstartswithSAP for audio tracks
([shaka-project#1346](https://github.com/joeyparrish/shaka-packager/issues/1346))
([d23cce8](d23cce8))
* Add support for ALAC codec
([shaka-project#1299](https://github.com/joeyparrish/shaka-packager/issues/1299))
([b68ec87](b68ec87))
* Add support for single file TS for HLS
([shaka-project#934](https://github.com/joeyparrish/shaka-packager/issues/934))
([4aa4b4b](4aa4b4b))
* Add support for the EXT-X-START tag
([shaka-project#973](https://github.com/joeyparrish/shaka-packager/issues/973))
([76eb2c1](76eb2c1))
* Add xHE-AAC support
([shaka-project#1092](https://github.com/joeyparrish/shaka-packager/issues/1092))
([5d998fc](5d998fc))
* Allow LIVE UDP WebVTT input
([shaka-project#1349](https://github.com/joeyparrish/shaka-packager/issues/1349))
([89376d3](89376d3))
* **DASH:** Add Label element.
([shaka-project#1175](https://github.com/joeyparrish/shaka-packager/issues/1175))
([b1c5a74](b1c5a74))
* **DASH:** Add video transfer characteristics.
([shaka-project#1210](https://github.com/joeyparrish/shaka-packager/issues/1210))
([8465f5f](8465f5f))
* default text zero bias
([shaka-project#1330](https://github.com/joeyparrish/shaka-packager/issues/1330))
([2ba67bc](2ba67bc))
* Drop Python 2 support in all scripts
([3e71302](3e71302))
* Generate the entire AV1 codec string when the colr atom is present
([shaka-project#1205](https://github.com/joeyparrish/shaka-packager/issues/1205))
([cc9a691](cc9a691)),
closes
[shaka-project#1007](https://github.com/joeyparrish/shaka-packager/issues/1007)
* HLS / DASH support forced subtitle
([shaka-project#1020](https://github.com/joeyparrish/shaka-packager/issues/1020))
([f73ad0d](f73ad0d))
* Move all third-party deps into git submodules
([shaka-project#1083](https://github.com/joeyparrish/shaka-packager/issues/1083))
([3e71302](3e71302))
* order streams in manifest based on command-line order
([shaka-project#1329](https://github.com/joeyparrish/shaka-packager/issues/1329))
([aad2a12](aad2a12))
* Parse MPEG-TS PMT ES language and maximum bitrate descriptors
([shaka-project#369](https://github.com/joeyparrish/shaka-packager/issues/369))
([shaka-project#1311](https://github.com/joeyparrish/shaka-packager/issues/1311))
([c09eb83](c09eb83))
* Portable, fully-static release executables on Linux
([shaka-project#1351](https://github.com/joeyparrish/shaka-packager/issues/1351))
([9be7c2b](9be7c2b))
* Replace glog with absl::log, tweak log output and flags
([3e71302](3e71302))
* Replace gyp build system with CMake
([3e71302](3e71302)),
closes
[shaka-project#1047](https://github.com/joeyparrish/shaka-packager/issues/1047)
* Respect the file mode for HttpFiles
([shaka-project#1081](https://github.com/joeyparrish/shaka-packager/issues/1081))
([3e71302](3e71302))
* This patch adds support for DTS:X Profile 2 audio in MP4 files.
([shaka-project#1303](https://github.com/joeyparrish/shaka-packager/issues/1303))
([07f780d](07f780d))
* Update all dependencies
([3e71302](3e71302))
* Write colr atom to muxed mp4
([shaka-project#1261](https://github.com/joeyparrish/shaka-packager/issues/1261))
([f264bef](f264bef)),
closes
[shaka-project#1202](https://github.com/joeyparrish/shaka-packager/issues/1202)


### Bug Fixes

* Accept 100% when parsing WEBVTT regions
([shaka-project#1006](https://github.com/joeyparrish/shaka-packager/issues/1006))
([e1b0c7c](e1b0c7c)),
closes
[shaka-project#1004](https://github.com/joeyparrish/shaka-packager/issues/1004)
* Add missing &lt;cstdint&gt; includes
([shaka-project#1306](https://github.com/joeyparrish/shaka-packager/issues/1306))
([ba5c771](ba5c771)),
closes
[shaka-project#1305](https://github.com/joeyparrish/shaka-packager/issues/1305)
* Add missing limits header
([efbca39](efbca39))
* Always log to stderr by default
([shaka-project#1350](https://github.com/joeyparrish/shaka-packager/issues/1350))
([35c2f46](35c2f46)),
closes
[shaka-project#1325](https://github.com/joeyparrish/shaka-packager/issues/1325)
* AudioSampleEntry size caluations due to bad merge
([shaka-project#1354](https://github.com/joeyparrish/shaka-packager/issues/1354))
([615720e](615720e))
* **CI:** Add Mac-arm64 to build matrix
([shaka-project#1359](https://github.com/joeyparrish/shaka-packager/issues/1359))
([c456ad6](c456ad6))
* **CI:** Add missing Linux arm64 builds to release
([9c033b9](9c033b9))
* dash_roles add role=description for DVS audio per DASH-IF-IOP-v4.3
([shaka-project#1054](https://github.com/joeyparrish/shaka-packager/issues/1054))
([dc03952](dc03952))
* Don't close upstream on HttpFile::Flush
([shaka-project#1201](https://github.com/joeyparrish/shaka-packager/issues/1201))
([53d91cd](53d91cd)),
closes
[shaka-project#1196](https://github.com/joeyparrish/shaka-packager/issues/1196)
* duplicate representation id for TTML when forced ordering is on
([shaka-project#1364](https://github.com/joeyparrish/shaka-packager/issues/1364))
([0fd815a](0fd815a)),
closes
[shaka-project#1362](https://github.com/joeyparrish/shaka-packager/issues/1362)
* duration formatting and update mpd testdata to reflect new format
([shaka-project#1320](https://github.com/joeyparrish/shaka-packager/issues/1320))
([56bd823](56bd823))
* Explicitly signal the lack of CEA captions in HLS
([d48bf0f](d48bf0f)),
closes [shaka-project#922](https://github.com/joeyparrish/shaka-packager/issues/922)
* Fix build errors related to std::numeric_limits
([shaka-project#972](https://github.com/joeyparrish/shaka-packager/issues/972))
([9996c73](9996c73))
* Fix build on FreeBSD
([shaka-project#1287](https://github.com/joeyparrish/shaka-packager/issues/1287))
([3e71302](3e71302))
* Fix clang build
([shaka-project#1288](https://github.com/joeyparrish/shaka-packager/issues/1288))
([3e71302](3e71302))
* Fix crash in static-linked linux builds
([e2d66b3](e2d66b3)),
closes [shaka-project#996](https://github.com/joeyparrish/shaka-packager/issues/996)
* Fix failure fetching encryption keys
([7392d80](7392d80))
* Fix failure on very short WebVTT files
([shaka-project#1216](https://github.com/joeyparrish/shaka-packager/issues/1216))
([dab165d](dab165d)),
closes
[shaka-project#1217](https://github.com/joeyparrish/shaka-packager/issues/1217)
* Fix handling of non-interleaved multi track FMP4 files
([shaka-project#1214](https://github.com/joeyparrish/shaka-packager/issues/1214))
([dcf3225](dcf3225)),
closes
[shaka-project#1213](https://github.com/joeyparrish/shaka-packager/issues/1213)
* Fix issues with `collections.abc` in Python 3.10+
([shaka-project#1188](https://github.com/joeyparrish/shaka-packager/issues/1188))
([80e0240](80e0240)),
closes
[shaka-project#1192](https://github.com/joeyparrish/shaka-packager/issues/1192)
* Fix local files with UTF8 names
([shaka-project#1246](https://github.com/joeyparrish/shaka-packager/issues/1246))
([3e71302](3e71302))
* Fix missing newline at the end of usage
([shaka-project#1352](https://github.com/joeyparrish/shaka-packager/issues/1352))
([6276584](6276584))
* Fix Python 3.10+ compatibility in scripts
([3e71302](3e71302))
* Fix tags in official Docker images and binaries
([73a85ce](73a85ce)),
closes
[shaka-project#1366](https://github.com/joeyparrish/shaka-packager/issues/1366)
* Fix uninitialized value found by Valgrind
([shaka-project#1336](https://github.com/joeyparrish/shaka-packager/issues/1336))
([7ef5167](7ef5167))
* Fix various build issues on macOS
([3e71302](3e71302))
* Fix various build issues on Windows
([3e71302](3e71302))
* hls, set the DEFAULT explicitly to NO. Supports native HLS players.
([shaka-project#1170](https://github.com/joeyparrish/shaka-packager/issues/1170))
([1ab6818](1ab6818)),
closes
[shaka-project#1169](https://github.com/joeyparrish/shaka-packager/issues/1169)
* http_file: Close upload cache on task exit
([shaka-project#1348](https://github.com/joeyparrish/shaka-packager/issues/1348))
([6acdcc3](6acdcc3)),
closes
[shaka-project#1347](https://github.com/joeyparrish/shaka-packager/issues/1347)
* Indexing `bytes` produces `int` on python3 for `pssh-box.py`
([shaka-project#1228](https://github.com/joeyparrish/shaka-packager/issues/1228))
([d9d3c7f](d9d3c7f)),
closes
[shaka-project#1227](https://github.com/joeyparrish/shaka-packager/issues/1227)
* Low Latency DASH: include the "availabilityTimeComplete=false"
attribute
([shaka-project#1198](https://github.com/joeyparrish/shaka-packager/issues/1198))
([d687ad1](d687ad1))
* misleading log output when HLS target duration updates (fixes
[shaka-project#969](https://github.com/joeyparrish/shaka-packager/issues/969))
([shaka-project#971](https://github.com/joeyparrish/shaka-packager/issues/971))
([f7b3986](f7b3986))
* **MP4:** Add compatible brand dby1 for Dolby content.
([shaka-project#1211](https://github.com/joeyparrish/shaka-packager/issues/1211))
([520926c](520926c))
* Parse one frame mpeg-ts video
([shaka-project#1015](https://github.com/joeyparrish/shaka-packager/issues/1015))
([b221aa9](b221aa9)),
closes
[shaka-project#1013](https://github.com/joeyparrish/shaka-packager/issues/1013)
* preserve case for stream descriptors
([shaka-project#1321](https://github.com/joeyparrish/shaka-packager/issues/1321))
([5d44368](5d44368))
* Prevent crash in GetEarliestTimestamp() if periods are empty
([shaka-project#1173](https://github.com/joeyparrish/shaka-packager/issues/1173))
([d6f28d4](d6f28d4)),
closes
[shaka-project#1172](https://github.com/joeyparrish/shaka-packager/issues/1172)
* PTS diverge DTS when DTS close to 2pow33 and PTS more than 0
([shaka-project#1050](https://github.com/joeyparrish/shaka-packager/issues/1050))
([ab8ab12](ab8ab12)),
closes
[shaka-project#1049](https://github.com/joeyparrish/shaka-packager/issues/1049)
* remove extra block assumptions in mbedtls integration
([shaka-project#1323](https://github.com/joeyparrish/shaka-packager/issues/1323))
([db59ad5](db59ad5)),
closes
[shaka-project#1316](https://github.com/joeyparrish/shaka-packager/issues/1316)
* Restore support for legacy FairPlay system ID
([shaka-project#1357](https://github.com/joeyparrish/shaka-packager/issues/1357))
([4d22e99](4d22e99))
* Roll back depot_tools, bypass vpython
([shaka-project#1045](https://github.com/joeyparrish/shaka-packager/issues/1045))
([3fd538a](3fd538a)),
closes
[shaka-project#1023](https://github.com/joeyparrish/shaka-packager/issues/1023)
* set array_completeness in HEVCDecoderConfigurationRecord correctly
([shaka-project#975](https://github.com/joeyparrish/shaka-packager/issues/975))
([270888a](270888a))
* TTML generator timestamp millisecond formatting
([shaka-project#1179](https://github.com/joeyparrish/shaka-packager/issues/1179))
([494769c](494769c)),
closes
[shaka-project#1180](https://github.com/joeyparrish/shaka-packager/issues/1180)
* Update golden files for ttml tests and failing hls unit tests.
([shaka-project#1226](https://github.com/joeyparrish/shaka-packager/issues/1226))
([ac47e52](ac47e52))
* Update to use official FairPlay UUID.
([shaka-project#1281](https://github.com/joeyparrish/shaka-packager/issues/1281))
([ac59b9e](ac59b9e))
* use a better estimate of frame rate for cases with very short first
sample durations
([shaka-project#838](https://github.com/joeyparrish/shaka-packager/issues/838))
([5644041](5644041))
* webvtt single cue do not fail on EOS
([shaka-project#1061](https://github.com/joeyparrish/shaka-packager/issues/1061))
([b9d477b](b9d477b)),
closes
[shaka-project#1018](https://github.com/joeyparrish/shaka-packager/issues/1018)

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions github-actions bot added the status: archived Archived and locked; will not be updated label Apr 8, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 8, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status: archived Archived and locked; will not be updated
Projects
None yet
3 participants