HLS encoded video (ffmpeg) breaks in Safari when segmented via byte-range

Question

Some of my HLS encoded videos via ffmpeg drop the audio when seeking past the buffer. The only way to sync the audio and video up again is to restart the video. What would be causing this?

Example Profile:

bitrate: 4800, profile: 'high', level: '4.1', resolution: 1080, framerate: '24000/1001'

ffmpeg
    '-y' 
    '-i' input_file.mov                                                 
    '-v' error 
    '-map' '0:0'                                                        
    '-c:v' libx264                                                  
    '-x264opts' f'
        keyint=23:                  
        min-keyint=23:          
        no-scenecut                                                     
    '
    '-vf' f'scale=-1:1080'
    '-preset' 'slow'
    '-profile:v' 'high'
    '-level' '4.1'
    '-b:v' '4800k'
    '-maxrate' '4800k'
    '-movflags' 'faststart'
    '-bufsize' '9600k' 
    '-write_tmcd', '0'
    '-r' '24000/1001'                                   
    output_dir

Segmentation CMD:

FFMPEG 
    '-i' output_dir 
    '-v' 'error' 
    '-acodec' 'copy'
    '-vcodec' 'copy'
    '-hls_time' '4' #seconds
    '-hls_list_size' '0'
    '-hls_flags' 'single_file' 
    os.path.join(output_dir, f'{run_id}_{bitrate}.m3u8'

Added: apple's mediastreamvalidator outputs a few different errors:

Error: Playlist vs segment duration mismatch
--> Detail:  Segment duration 98.0146, Playlist duration: 4.7968
--> Source:  1559962503399_2200k.m3u8 - 1559962503399_2200k.ts:1746520@0

Error: Measured peak bitrate compared to master playlist declared value exceeds error tolerance
--> Detail:  Measured: 3182.61 kb/s, Master playlist: 2173.82 kb/s, Error: 46.41%, Combined rendition name: English
--> Source:  ...playlist.m3u8
--> Compare: 1559962503399_2200k.m3u8

Error: Different target durations detected
--> Detail:  Target duration: 5 vs Target duration: 4
--> Source:  1559962503399_64k.m3u8
--> Compare: 1559962503399_128k.m3u8

UPDATE 1: I modified the encoding command to utilize tee pseudo-muxer, but it seems the same issue still exists when segmenting HLS as byte-ranges rather than in separate .ts files:

$ ffmpeg
-hide_banner
-report
-benchmark
-vstats
-i "../Jane_shallowing_Top_v08.mp4"
-dn
-sn
-filter_complex "[0:v]fps=fps=24.000,setpts=(PTS-STARTPTS),split=[vsplit1][vsplit2];[vsplit1]scale=-1:144[video_144];[vsplit2]scale=-1:1080[video_1080]"
-map "[video_144]"
-r:v:0 "24.000"
-c:v:0 "libx264"
-x264-params "keyint=144:min-keyint=144:scenecut=0:open_gop=0"
-preset:v:0 "slow"
-profile:v:0 "baseline"
-refs:v:0 "2"
-b-pyramid:v:0 "strict"
-tune:v:0 "film"
-b:v:0 "96000"
-maxrate:v:0 "56000"
-bufsize:v:0 "6*56000/8"
-vsync:v:0 "cfr"
-bsf:v:0 "h264_metadata=fixed_frame_rate_flag=1"
-map "[video_1080]"
-r:v:1 "24.000"
-c:v:1 "libx264"
-x264-params "keyint=144:min-keyint=144:scenecut=0:open_gop=0"
-preset:v:1 "slow"
-profile:v:1 "high"
-refs:v:1 "2"
-b-pyramid:v:1 "strict"
-tune:v:1 "film"
-b:v:1 "4800000"
-maxrate:v:1 "4800000"
-bufsize:v:1 "6*4800000/8"
-vsync:v:1 "cfr"
-bsf:v:1 "h264_metadata=fixed_frame_rate_flag=1"
-map a:0 -map a:0
-c:a "libfdk_aac"
-ar "48000"
-ab "128k"
-af "aresample=async=1:min_hard_comp=0.100000:first_pts=0"
-f "hls"
-var_stream_map "v:1,a:0 v:0,a:1"
-hls_time "6.000"
-hls_segment_type "mpegts"
-hls_flags "discont_start+temp_file+single_file"
-hls_list_size "0"
-master_pl_name "playlist.m3u8"
-hls_segment_filename "out_%v.ts" "out_%v.m3u8"

Segment duration 98.0267, Playlist duration: 6.0000

Is there any (specific) reason you're not encoding and segmenting at the same time? What you're doing here with an intermediate file seems unnecessary, unless there are other workflow requirements not clarified in your question. If that's the case, care to elaborate? — Dennis Mungai, Jun 13 '19 at 18:23
The workflow of this inherited project is as follows: encode the audio, encode video and segment at each bitrate (400k, 3200k, etc.), mp4box to mpd, transform the dash playlist to get hls playlist, then encode some videos again at base resolutions (1920x1280, 1080x720, etc.). Is there a better way? (no I do not believe there is a specific reason other than having to do this process for each bitrate profile?) — eschie, Jun 13 '19 at 20:03
Also, I think the original rendered input video contains I frames already. How can I analyze this? — eschie, Jun 13 '19 at 20:03
yes, there's a better way. 1:N transcoding is an option, then switch to the tee muxer to handle multiple outputs with stream re-use. — Dennis Mungai, Jun 14 '19 at 13:43
@林正浩 I updated to use the tee muxer (see above), but the issue persists. I feel like segment and playlist duration are switched as before? Also, this doesn't group the audio track separately, is there a benefit to having it separate? — eschie, Jun 15 '19 at 02:11

score 1 · Answer 1 · answered Jun 13 '19 at 02:31

1

You have to set the GOP based on the framerate and the segment time. In your example you have 24 frames per second and 4 second segments. The GOP should be set to 96.

Also try setting minrate maxrate and bufsize the same. If this works, try extending the bufsize to what you want.

I would suggest something like (Notice how the GOP is 24(Framerate)*4(HLS Container Time);

ffmpeg -i videofile -c:v libx264 -preset slow -profile:v high -level 4.1 -minrate 4800k -maxrate 4800k -bufsuze 4800k -b:v 4800k -vf scale=-1:1080 -flags +cgop -g 96 -x264-params scenecut=0:open_gop=0:min-keyint=96:keyint=96

You mention nothing of audio chain. I would suggest something like;

-c:a aac -b:a 64k -filter:a aresample=48000:async=1:min_hard_comp=0.100000:first_pts=0 -fflags +genpts

Change out the audio codec if you have libfdk_aac and other settings as needed. The key settings are the correct GOP for the segment size.

answered Jun 13 '19 at 02:31

Geoff King

11
1

Thanks Geoff. I tried this but it seems to have made things worse? Now the video files all have individually different target durations, and in the native player, the video length is about 10 times as large in the playback menu. And seeking forward to any point just restarts the video. Maybe it has something to do with my segmentation command? Most of the first segments for each profile are much larger than then the target duration of 4s. Here's my command: – eschie Jun 13 '19 at 05:35
`FFMPEG -y -i input_file.mov -v error -map 0:0 -c:v libx264 -x264-params keyint=96:min-keyint=96:scenecut=0:open_gop=0 -vf scale=-1:1080 -preset slow -profile:v high -level 4.1 -b:v 4800k -minrate 4800k -maxrate 4800k -bufsize 4800k -flags +cgop -g 96 -movflags faststart intermediate_file.mp4` – eschie Jun 13 '19 at 05:39
```bitrate: 3200, profile: main, lvl: 4.1, res: 720, framerate: 24000/1001 bitrate: 2200, profile: main, lvl: 3.1, res: 720, framerate: 24000/1001 bitrate: 1400, profile: main, lvl: 3.1, res: 540, framerate: 24000/1001 bitrate: 900, profile: main, lvl: 3.1, res: 432, framerate: 24000/1001 bitrate: 400, profile: baseline, lvl: 3.0, res: 270, framerate: 24000/1001 bitrate: 200, profile: main, lvl: 3.1, res: 270, framerate: 24000/1001 bitrate: 64, profile: high, lvl: 4.1, res: 144, framerate: 10000/1001 bitrate: 56, profile: baseline, lvl: 3.0, res: 144, framerate: 10000/1001``` – eschie Jun 13 '19 at 05:42

HLS encoded video (ffmpeg) breaks in Safari when segmented via byte-range

1 Answers1