Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LIVE packaging for low latency #744

Closed
sridhard opened this issue Apr 4, 2020 · 1 comment
Closed

LIVE packaging for low latency #744

sridhard opened this issue Apr 4, 2020 · 1 comment
Labels
status: archived Archived and locked; will not be updated status: duplicate A duplicate of another issue; should be closed after linking to the original issue

Comments

@sridhard
Copy link

sridhard commented Apr 4, 2020 •
edited
Loading

System info

Operating System: <e.g. macOS Sierra, Ubuntu 14.04 trusty etc>
Ubuntu 18

Shaka Packager Version: <e.g. v1.6.1, commit SHA etc>
2.4.1

Issue and steps to reproduce the problem

I am using ffmpeg+shaka-packager for LIVE streaming. Currently the latency is around 20 sec. To reduce the latency I am facing some issues:

  1. To reduce the latency one of the important recommendation in low latency live stream, recommended settings? shaka-player#1287 is using SegmentTemplate with duration. I checked Segment Template with Duration #400 and as per this we need to use --allow_approximate_segment_timeline --segment_template_constant_duration. But even after that I am not able to generate mpd with duration. To make sure that there is no audio and video alignment issues while generating segments I have created mpd with only video segments. Below are the ffmpeg and packager commands:

ffmpeg -y -listen 1 -thread_queue_size 200 -i "rtmp://127.0.0.1:1437/live/abc" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:640,setsar=1:1 -f mpegts "udp://127.0.0.1:40000" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:360,setsar=1:1 -f mpegts "udp://127.0.0.1:40001"

packager "in=udp://127.0.0.1:40000,stream=video,init_segment=./livedata/HQVideo_init.mp4,segment_template=./livedata/HQVideo_$Number$.m4s" --segment_duration 1 "in=udp://127.0.0.1:40001,stream=video,init_segment=./livedata/MQVideo_init.mp4,segment_template=./livedata/MQVideo_$Number$.m4s" --segment_duration 1 --mpd_output "./livedata/$1/$2/$3/livestream1.mpd" --allow_approximate_segment_timeline --segment_template_constant_duration

FFMPEG generate 2 video streams which are sent to shaka-packager with udp. Below is the mpd generated:

<?xml version="1.0" encoding="UTF-8"?>
<!--Generated with https://github.com/google/shaka-packager version v2.4.1-c731217-release-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-live:2011" minBufferTime="PT2S" type="dynamic" publishTime="2020-04-04T16:37:17Z" availabilityStartTime="2020-04-04T16:36:44Z" minimumUpdatePeriod="PT5S" timeShiftBufferDepth="PT1800S">
  <Period id="0" start="PT0S">
    <AdaptationSet id="0" contentType="video" maxWidth="1024" maxHeight="640" frameRate="90000/3000" par="8:5">
      <Representation id="0" bandwidth="885368" codecs="avc1.42c01e" mimeType="video/mp4" sar="1:1" width="1024" height="640">
        <SegmentTemplate timescale="90000" initialization="MV1RRUYQ7Q6D/HQVideo_init.mp4" media="MV1RRUYQ7Q6D/HQVideo_$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="126000" d="90000" r="31"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="1" bandwidth="355728" codecs="avc1.42c01e" mimeType="video/mp4" sar="1:1" width="576" height="360">
        <SegmentTemplate timescale="90000" initialization="MV1RRUYQ7Q6D/MQVideo_init.mp4" media="MV1RRUYQ7Q6D/MQVideo_$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="126000" d="90000" r="31"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

As you can see there is no segment-duration and still it has SegmentTimeline. Because of this periodic manifest updates are required.

  1. If I add the audio stream then there is another issue. As per this MPD segment length issue #330 we need to align audio and video segments, but I a not able to achieve it:

ffmpeg -y -listen 1 -thread_queue_size 200 -i "rtmp://127.0.0.1:1437/live/abc" -vn -ac 1 -c:a aac -b:a 128k -r 44100 -f mpegts -strict experimental "udp://127.0.0.1:40000" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:640,setsar=1:1 -f mpegts "udp://127.0.0.1:40001" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:360,setsar=1:1 -f mpegts "udp://127.0.0.1:40002"

packager "in=udp://127.0.0.1:40000,stream=audio,init_segment=./livedata/audio_init.mp4,segment_template=./livedata//audio_$Number$.m4s" --segment_duration 1 "in=udp://127.0.0.1:40001,stream=video,init_segment=./livedata/HQVideo_init.mp4,segment_template=./livedata/HQVideo_$Number$.m4s" --segment_duration 1 "in=udp://127.0.0.1:40002,stream=video,init_segment=./livedata/MQVideo_init.mp4,segment_template=./livedata/MQVideo_$Number$.m4s" --segment_duration 1 --mpd_output "./livedata/$1/$2/$3/livestream1.mpd" --allow_approximate_segment_timeline --segment_template_constant_duration

Below is the MPD I am getting:


<?xml version="1.0" encoding="UTF-8"?>
<!--Generated with https://github.com/google/shaka-packager version v2.4.1-c731217-release-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-live:2011" minBufferTime="PT2S" type="dynamic" publishTime="2020-04-04T17:25:43Z" availabilityStartTime="2020-04-04T17:25:20Z" minimumUpdatePeriod="PT5S" timeShiftBufferDepth="PT1800S">
  <Period id="0" start="PT0S">
    <AdaptationSet id="0" contentType="video" maxWidth="1024" maxHeight="640" frameRate="90000/3000" par="8:5">
      <Representation id="0" bandwidth="927280" codecs="avc1.42c01e" mimeType="video/mp4" sar="1:1" width="1024" height="640">
        <SegmentTemplate timescale="90000" initialization="5XQIY6VQAWLC/HQVideo_init.mp4" media="5XQIY6VQAWLC/HQVideo_$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="126000" d="90000" r="21"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="1" bandwidth="384936" codecs="avc1.42c01e" mimeType="video/mp4" sar="1:1" width="576" height="360">
        <SegmentTemplate timescale="90000" initialization="5XQIY6VQAWLC/MQVideo_init.mp4" media="5XQIY6VQAWLC/MQVideo_$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="126000" d="90000" r="21"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
    <AdaptationSet id="1" contentType="audio" segmentAlignment="true">
      <Representation id="2" bandwidth="145652" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="44100">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="1"/>
        <SegmentTemplate timescale="90000" initialization="5XQIY6VQAWLC/audio_init.mp4" media="5XQIY6VQAWLC/audio_$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="126000" d="54332"/>
            <S t="180332" d="89817" r="1"/>
            <S t="359966" d="91996"/>
            <S t="451962" d="89861" r="13"/>
            <S t="1710016" d="91954"/>
            <S t="1801970" d="89861" r="3"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

Packager Command:

Extra steps to reproduce the problem?
All the details I have given above

What is the expected result?
shaka-packager should generate mpd with SegmentTemplate with duration so that the mpd is small and also mpd updates are not required as per shaka-project/shaka-player#1287

What happens instead?
shaka-packager is generating mpd with segment timelines.

<Please attach the input files or email to shaka-packager-issues@google.com.>
@sridhard sridhard mentioned this issue Apr 5, 2020
Open
@kqyang
Copy link
Contributor

kqyang commented Apr 6, 2020

@sridhard This is the same issue as #554. Let's track there.

In addition to approximate timeline support, we'll need additional work as mentioned in #675 to support CMAF low latency chunks.

@kqyang kqyang closed this as completed Apr 6, 2020
@kqyang kqyang added status: duplicate A duplicate of another issue; should be closed after linking to the original issue and removed needs triage labels Apr 6, 2020
@shaka-bot shaka-bot added the status: archived Archived and locked; will not be updated label Jun 5, 2020
@shaka-project shaka-project locked and limited conversation to collaborators Jun 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
status: archived Archived and locked; will not be updated status: duplicate A duplicate of another issue; should be closed after linking to the original issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sridhard @kqyang @shaka-bot