0

I'm using ffmpeg to split up a video by frames. For a video that is 25fps, my expectation would be that it would show frame 0 for 40ms, and then it would show frame 1 for 40ms, etc. However, when I use ffmepg to analyze the video, this is not what I get.

I pulled frames out of the video using the following commands:

ffmpeg -i original.mp4 -ss 00:00:00.000 -vframes 1 0.000.png
ffmpeg -i original.mp4 -ss 00:00:00.001 -vframes 1 0.001.png
ffmpeg -i original.mp4 -ss 00:00:00.040 -vframes 1 0.040.png
ffmpeg -i original.mp4 -ss 00:00:00.041 -vframes 1 0.041.png

In this, I would expect 0.000.png and 0.001.png to be the same frame, but the resulting pngs are different. However, on the flipside, 0.001.png and 0.040.png ARE the same frame, while 0.040.png and 0.041.png are different frames. It seems as though the frame at 0.000.png is only showing for 1ms. I've tested this on multiple different videos, with consistent results. Can anybody explain why this is or why my expectations are potentially incorrect?

As a side note, when I load the same video into an HTML5 video tag in the browsers, when I play the video the first frame shows for 64 milliseconds, and all other frames show for 40 milliseconds. I can test this by comparing frames for the following commands:

player.currentTime = 0
player.currentTime = .063 // same frame as 0
player.currentTime = .064 // different frame than .063
player.currentTime = .103 // same frame as .064
player.currentTime = .104 // different frame than .103

This also defies expectations, and makes matching up frames/times for the video between ffmpeg and html5 video even more annoying. Does anyone know what's going on here?

If it's helpful, here is the metadata from the video:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'original.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.17.101
  Duration: 01:36:16.20, start: 0.000000, bitrate: 2301 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2200 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Chubacca
  • 435
  • 1
  • 5
  • 9

1 Answers1

1

-ss will match the first frame at or after the specified timestamp.

As for the first frame duration and subsequent video frame timestamps, your file has a AAC audio track, which has a priming sample, and looks like the browser/player does not skip the priming sample, and offsets video timestamps to preserve sync.

Gyan
  • 85,394
  • 9
  • 169
  • 201
  • Thanks for the response - the second part makes sense. Can you elaborate on the first part? Is the implication that the frame 0 is "at" the 0ms mark, and frame 1 is "at" the 40ms mark, so at 1ms there is no frame there, so we go to the next frame (frame 1)? That's how I'm understanding your comment. – Chubacca Apr 13 '19 at 20:58
  • Frames have a presentation timestamp - the time when it is painted on the output - e.g. player window. The drawn frame will likely be refreshed by the display driver till the next frame painting event by the player, but isn't part of the frame "time". The timestamp is a 'point' metric, not a 'span'. – Gyan Apr 14 '19 at 04:49