0

This video is created using ffmpeg command from a series of images with number and expected timestamp on it (assuming 120 fps constant frame rate):

ffmpeg -framerate 120 -i %03d.png -crf 0 -x264opts keyint=1:min-keyint=1 120.mp4

It's supposed to be constant frame rate, and all the frames are I frames. I expect frame 6 will show up exactly at time 0.05 second [0.05, 0.0583333), as the pts_time shown in ffmpeg and ffprobe:

[Parsed_showinfo_0 @ 0x7fba11404080] n: 6 pts: 768 pts_time:0.05 pos: 36021 fmt:yuv444p sar:0/1 s:1552x878 i:P iskey:1 type:I checksum:2CDED07A plane_checksum:[A86DBAAD C4CB8EF2 DDC586CC] mean:[163 133 121 ] stdev:[9.0 0.6 0.8 ]

[FRAME]
media_type=video
stream_index=0
key_frame=1
pkt_pts=768
pkt_pts_time=0.050000
pkt_dts=768
pkt_dts_time=0.050000
best_effort_timestamp=768
best_effort_timestamp_time=0.050000
pkt_duration=128
pkt_duration_time=0.008333
pkt_pos=36021
pkt_size=2531
width=1552
height=878
pix_fmt=yuv444p
sample_aspect_ratio=N/A
pict_type=I
coded_picture_number=6
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=unknown
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=left
[/FRAME]

But in reality, it shows up since 0.049943 second. I use ffmpeg to extract that time with 0.0001 second duration

ffmpeg -ss 00:00:00.049943 -i 120_keyint_1.mp4 -t 00:00:00.0001 out.mp4

and get this video, which shows frame 6.

I also tried playing this video on chrome, and set the currentTime of videoNode. It shows frame 5 at 0.049943, but shows frame 6 since 0.049968 (before 0.05!).

However, I tried another encoding method and get another constant frame rate video:

ffmpeg -framerate 120 -i %03d.png -vcodec libx264 -f mp4 -movflags faststart 120.mp4

theoretically frame 6 should also appear in time range [0.05, 0.0583333), but in Chrome frame 6 will not show up until currentTime >= 0.051 second. More interestingly, I get frame 6 at 0.049943 second using ffmpeg, the same extraction command above.

Does this mean in MPEG4 h.264 video, we cannot rely on pts_time from ffmpeg to determine exact show time of a specific frame and it's video player specific?

Lincolnhuj
  • 63
  • 6
  • 1
    Yes, some jitter is expected. Curious, what's your application that requires microsecond precision? – Gyan Sep 17 '20 at 14:37
  • My web app is for online video editing and require frame accuracy. We're looking for a video format that can have strict frame to timestamp mapping. – Lincolnhuj Sep 17 '20 at 14:43
  • Actual video playback imprecision has no impact on how you store or manipulate timestamps in software or file data structures. – Gyan Sep 17 '20 at 15:49

1 Answers1

2

I think the issue is the Movie Header scale of 1000 in your file which is the hard wired default of ffmpeg. (libavformat\movenc.h line 32: #define MOV_TIMESCALE 1000)

Your frame rate of 120 fps can't be express in 1/1000s - so ffmpeg produces rounding errors.

1000 as default time scale is a poor choice and it can't be changed without modifying ffmpeg's source code.

I would change #define MOV_TIMESCALE 1000 to #define MOV_TIMESCALE 600 or some other value that is divisible by 120 and recompile ffmpeg.

Markus Schumann
  • 7,636
  • 1
  • 21
  • 27
  • 1
    MOV/MP4s store sample duration per-sample (chunks, actually), so framerate doesn't enter into it, not to mention the PTS of the relevant frame can be exactly represented in a scale of 1000 as 50. OP is complaining about imprecision during playback, not ffmpeg. – Gyan Sep 17 '20 at 15:50
  • @Gyan ffmpeg extraction also gives wrong result. `ffmpeg -ss 00:00:00.049943 -i 120_keyint_1.mp4 -t 00:00:00.0001 out.mp4` gives a clip with frame 6, but should be frame 5 – Lincolnhuj Sep 17 '20 at 22:51
  • Why should it be frame 5? – Gyan Sep 18 '20 at 04:34
  • @Gyan Note the frame starts from 0. The pts_time of frame 6 is 0.05 second, according to ffmpeg "showinfo". So at 0.049943 second it should be frame 5. – Lincolnhuj Sep 18 '20 at 05:31
  • 1
    Seeking doesn't work that way. The relative timestamps and packet durations will be demuxed as stored, not adjusted for intermediate time intervals. Further, seeking for MP4 normally seeks backwards to the nearest keyframe. In your case, however it seeks to frame 6 which is a KF. That's because `0.049943` when converted to the stream timebase (1/15360) is 767.12448 which gets rounded towards +infinity resulting in 768. And frame 6 is a KF exactly at pts 768. – Gyan Sep 18 '20 at 09:59
  • Thanks for explaining! @Gyan Is it always rounding towards +infinity? How can we get this rounding method from ffmpeg? – Lincolnhuj Sep 18 '20 at 12:46
  • Asked a separate question here https://stackoverflow.com/questions/63956476/given-a-constant-frame-rate-h-264-mp4-how-to-get-pts-rounding-method-used-durin – Lincolnhuj Sep 18 '20 at 13:15