Bug: HLS stream URLs get truncated in ffmpeg

[leftearsons]

Observed Behavior

It is observed that in some cases, the internal ffmpeg library, truncates the HLS URL which causes valuable authentication data to be discarded. This leads to invalid video stream urls and eventually to stopping HLS playback.
By looking at the logs, the issue seems to take place inside the 'https' thread, since the 'hls' thread always logs the correct (full length) URL path.

The pattern of the failing URLs makes me think that the root cause of this case, could be the query parameter length of the HLS URL, since it is only observed when the URL contains the authentication data in query parameters, instead of inside the URL path.
We have been loading HLS video streams from the same video providers for a long time and no change has been made on the video source, meaning that the same HLS URLs format used to work well with CCG v2.07.

I am adding for your reference the log files for those two cases, a faulty one (in which the URL contains path parameters) and a successful one (in which the URL contains only path parameters).

Faulty case:

[info]    Received message from 127.0.0.1: PLAY 1-50 "https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k.m3u8?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad77c74e068ac5e898cdf8e7a270c3ffc30f62ba160c5b8efa5856c9"\r\n
[info]    Sent message to 127.0.0.1:202 PLAY OK\r\n
[info]    [ffmpeg] [hls @ 0000022E3604E300] Skip ('#EXT-X-VERSION:3')
[info]    
[info]    [ffmpeg] [hls @ 0000022E3604E300] Opening 'https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k_1718772001_0000690.ts?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad77c74e068ac5e898cdf8e7a270c3ffc30f62ba160c5b8efa5856c9' for reading
[info]    
[info]    [ffmpeg] [hls @ 0000022E3604E300] Opening 'https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k_1718772001_0000691.ts?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad77c74e068ac5e898cdf8e7a270c3ffc30f62ba160c5b8efa5856c9' for reading
[info]    
[info]    [ffmpeg] [https @ 0000022E328F2040] Opening 'https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k_1718772001_0000692.ts?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad' for reading
[info]    
[warning] ffmpeg[https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k.m3u8?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad77c74e068ac5e898cdf8e7a270c3ffc30f62ba160c5b8efa5856c9|0.0000/0.0000] Latency: 41
[warning] [ffmpeg] [https @ 0000022E5DEA41C0] HTTP error 403 Forbidden
[warning] 
[warning] [ffmpeg] [hls @ 0000022E3604E300] keepalive request failed for 'https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k_1718772001_0000692.ts?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad' with error: 'Server returned 403 Forbidden (access denied)' when opening url, retrying with new connection
[warning] 
[info]    [ffmpeg] [hls @ 0000022E3604E300] Opening 'https://lc-live-http-ipv4.akamaized.net/30714/4830385/desktop/table_tennis_international_tt_elite_series_retail_4830385_1364k_1718772001_0000692.ts?cid=30714&mid=50853377&ecid=4830385&pid=1&dtid=1&sid=848252974901&gc=pu4&gsd=oA&grm=1&hdnts=ip=85.73.218.100~exp=1718859799~acl=%2F30714%2F4830385%2Fdesktop%2F%2A~hmac=d3766115ad' for reading
[info]    
[warning] [ffmpeg] [https @ 0000022E57BF8D40] HTTP error 403 Forbidden
[warning] 
[warning] [ffmpeg] [hls @ 0000022E3604E300] Failed to open segment 692 of playlist 0

Successful case:

[info]    Received message from 127.0.0.1: PLAY 1-50 "https://744133.akamaized.net/hls/live/716232/ch12/89be75f87f0cbc976470fc2feb7ddd34db9a5390/video-m5_video/hdntl=exp=1719398046~acl=%2f*~id=668b56aecfe14366a26602c686807954~data=hdntl~hmac=be026824432c7fdbeb164ebe0cf92d159a38e2e18191633937b89732a8367be1/video.m3u8"\r\n
[info]    Sent message to 127.0.0.1:202 PLAY OK\r\n
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Skip ('#EXT-X-VERSION:3')
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Skip ('#EXT-X-INDEPENDENT-SEGMENTS')
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-06-25T10:34:00.000Z')
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-06-25T10:34:02.000Z')
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-06-25T10:34:04.000Z')
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Opening 'https://744133.akamaized.net/hls/live/716232/ch12/89be75f87f0cbc976470fc2feb7ddd34db9a5390/video-m5_video/hdntl=exp=1719398046~acl=%2f*~id=668b56aecfe14366a26602c686807954~data=hdntl~hmac=be026824432c7fdbeb164ebe0cf92d159a38e2e18191633937b89732a8367be1/4/file2357.ts' for reading
[info]    
[info]    [ffmpeg] [hls @ 000001DB7B8BE0C0] Opening 'https://744133.akamaized.net/hls/live/716232/ch12/89be75f87f0cbc976470fc2feb7ddd34db9a5390/video-m5_video/hdntl=exp=1719398046~acl=%2f*~id=668b56aecfe14366a26602c686807954~data=hdntl~hmac=be026824432c7fdbeb164ebe0cf92d159a38e2e18191633937b89732a8367be1/4/file2358.ts' for reading
[info]    
[info]    [ffmpeg] [https @ 000001DB7BA9A2C0] Opening 'https://744133.akamaized.net/hls/live/716232/ch12/89be75f87f0cbc976470fc2feb7ddd34db9a5390/video-m5_video/hdntl=exp=1719398046~acl=%2f*~id=668b56aecfe14366a26602c686807954~data=hdntl~hmac=be026824432c7fdbeb164ebe0cf92d159a38e2e18191633937b89732a8367be1/4/file2359.ts' for reading
[info]    
[warning] ffmpeg[https://744133.akamaized.net/hls/live/716232/ch12/89be75f87f0cbc976470fc2feb7ddd34db9a5390/video-m5_video/hdntl=exp=1719398046~acl=%2f*~id=668b56aecfe14366a26602c686807954~data=hdntl~hmac=be026824432c7fdbeb164ebe0cf92d159a38e2e18191633937b89732a8367be1/video.m3u8|0.0000/0.0000] Latency: 39

Expected behaviour

When loading an HLS stream, we would expect CasparCG to start the playback of it's chunks without altering their URL, eventually causing validation errors on the video streaming server.

Steps to reproduce

1. Load an HLS video stream with PLAY AMCP command, which uses long list of query parameters with authentication tokens
2. Observe the CasparCG logs and check for truncated HLS video stream URL when comparing the hls ffmpeg thread and the https ffmpeg thread logs, which would eventually cause HTTP validation errors (403 in our case)

Environment

• Server version 2.4.0 1e25c7a Stable
• Operating system: Windows 10

Issue was not present in version 2.07