11

I am trying to get representative frames for a video, so as to remove redundant frames which might appear in a video. This is what I use to get the frames.

./ffmpeg -i video.mp4 -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 2 -s 320x240 thumb-%02d.png

I have also tried

./ffmpeg -i video.mp4 -f image2 -vf "select=gt(scene\,.4)" -vsync vfr thumb%04d.png

The major issue in this is blur. If I just sample frames every 5 seconds, I don't see any blur, however using the above two commands I get a lot of blur.

The video can be found here, http://www.cs.umd.edu/~bharat/video.mp4

In order to sample the video every 10 seconds, I use the following:

./ffmpeg -i video.mp4 -r 1/10 filename%03d.jpg

Output using normal sampling: sampling output

Output using select: select output

However, normal sampling may be bad for some videos, and may create redundant frames. Is there a way I could use some option in ffmpeg and get frames without this blur? If normal sampling can get good frames, there should be frames in the vicinity without blur. I have looked at options such as scenecut in ffmpeg however I am not familiar with using them for this application.

Michael Kohne
  • 11,888
  • 3
  • 47
  • 79
Bharat
  • 2,139
  • 2
  • 16
  • 35
  • This result is expected with regular frame sampling. Does it necessarily need to be `ffmpeg`? There are plenty of freely available shot detection implementation. – bjou Jul 22 '14 at 19:05
  • not necessarily, anything will do, I just need to get rid of the blurry frames – Bharat Jul 22 '14 at 19:14

1 Answers1

3

I suggest also taking a look at other freely available shot detection implementations. For example, a custom threshold of 47 with Johan Mathe's Shotdetect yields the following results (first frame of every shot): Shot Detection with jmathe's Shotdetect

However, your question seems to deal more with the problem of video stabilization than it does scene change or shot detection. The blur that you see above as well as in the OP are "artifacts" inherent to the video and are not due to the algorithms used to cut or sample the video. If you want to reduce the amount of motion you should look into this post as well as look at OpenCV's Video Stabilization module. There is also a lot of research addressing this challenge including this and this.

bjou
  • 1,107
  • 1
  • 7
  • 19
  • thanks for the link, I understand that there is blur inherently present in the video, what was surprising that ffmpeg was selecting the blurry frames only when you use the select option, your results seem to be better, probably there is maximum variation in the frames selected by ffmpeg. – Bharat Jul 22 '14 at 23:20
  • I'm not sure what you mean by "maximum variation," but you may also want to experiment with what frame you select. The command you gave selects the I-Frame, but you can also select any of the following: `I, P, B, S, SI, SP, BI`. – bjou Jul 23 '14 at 00:22
  • I meant too blurry frames would look different from the normal frames and ffmpeg might detect it as a change in shot, it would be interesting to see what P,B frames give, thanks again, I did not know we could also select other frames. If I remember correctly there should be many P,B frames, how would it know which P/B frame to select – Bharat Jul 23 '14 at 00:56
  • That would be experimental. Sorry. Wish there was a better answer. – bjou Jul 23 '14 at 05:55
  • 3
    shotdetect link is broken – Michael Mar 21 '20 at 03:45