0

Question

I am trying to figure out how to sample a video stream x[n] every N frames, starting at frame n_i, i < N, so that I end up with N new videos of length len(x) / N.

In formula this is simply: y_i[n] = x[n_i + n * N].

Here there is a diagram of what I am trying to achieve: sampling

The greedy solution would be simply dumping the frames to a folder and then create new videos out of appropriately indexed frames. I was hoping there were some more elegant solution with ffmpeg since I have to process hundreds of video.

Implementation

Finally, I managed to write the final implementation, which I am reporting here for completeness.
It does scale the minimum dimension to 256, does not process more than max_frames; performs the sampling every k frames, send the first k - 1 samples to one folder and the k-th one to another one. It also set the output frame rate to the input average frame rate, since otherwise some videos will go at 120 Hz...

k=5
kk=$(awk "BEGIN{print 1/$k}")
ffmpeg \
    -i $src_video_path \
    -an \
    -loglevel error \
    -filter_complex \
        "setpts=$kk*PTS, \
        scale=w=2*trunc(128*max(1\, iw/ih)):h=2*trunc(128*max(1\, ih/iw))[m]; \
        [m]select=n=$k:e=(mod(n\,$k)+1)*lt(n\,$max_frames) \
        $(for ((i=1; i<=$k; i++)); do
            echo -n "[a$i]"
        done)" \
    $(for ((i=1; i<$k; i++)); do
        echo -n "-r $fps -map [a$i] $dst_video_path/$i.mp4 "
    done
    echo -n "-r $fps -map [a$k] $val_video_path/$k.mp4"
    )
Atcold
  • 683
  • 2
  • 10
  • 31

1 Answers1

4

One-by-one solution

You can use ffmpeg's select filter, which allows you to evaluate an expression using the sequential frame number, and decide whether or not to skip it (and which output to send it to if it isn't skipped).

To generate, for example, the second video where you are splitting into four each time, you would do the following:

ffmpeg -i input_video -vf select='not(mod(n-1\,4)), setpts=0.25*PTS' -an output_video

Change the 4 if you are splitting into a different number, and the 1 if you want a different selection (so for four you would run it four times, starting without subtracting anything, and ending with subtracting three.

The -vf ... -an segment is to get the video to play back at the right speed, and have the right length (which requires removing any audio tracks).

All-at-once solution

The select filter can also split the file and save each output separately. Here is a command that will split the file into k outputs, (change .mp4 if you want a different extension).

k=4;ffmpeg -i INPUT_FILE -an -filter_complex "setpts=`bc -l <<< 1/$k`*PTS[m];[m]select=n=$k:e=mod(n\,$k)+1`for ((i=1; i<=$k; i++)); do echo -n "[a$i]"; done`" `for ((i=1; i<=$k; i++)); do echo -n "-map [a$i] out$i.mp4 "; done`

The loops within the command are just to write out [a1][a2]... and -map [a1] out1.mp4.... Other than that the modification is to use filter_complex, which is required when dealing with multiple outputs, and changing the filter to direct each frame to its own output mod k.

gandaliter
  • 9,863
  • 1
  • 16
  • 23
  • `n-1`, `n+1`? Does it matter? Why are you subtracting? Summing was more logical, for me. I guess it does not change anything. – Atcold Mar 30 '17 at 22:16
  • 1
    If you subtract then the numbers match up with order of the outputs. If you add then you get them in reverse order. – gandaliter Mar 30 '17 at 22:22
  • 1
    In the one-by-one solution, the 2nd VF will override the first.You have to chain the filters. – Gyan Mar 31 '17 at 04:54
  • I've just tried the first line you posted. I get frames `1 5 5 9 13 17 21 ...`. So, I think there is still some issues – Atcold Mar 31 '17 at 12:30
  • So, everything works quite fine, but the second frame, for every video, is replicated. So, I have `0 4 4 8 ...`, `1 5 5 9 ...`, `2 6 6 10 ...` and `3 7 7 11 ...`. VLC does not like the video length and does not let me step frame-by-frame `240` of `292`, but this is not an issue for the moment. (My testing video has `294` frames.) – Atcold Mar 31 '17 at 12:50
  • I think there is a numerical problem here between PTS and FPS. To me it would be just fine having both PTS and FPS set to 1, so that duplication would not occur. But, perhaps, I did not get what is going on. – Atcold Mar 31 '17 at 13:03
  • Last question. How do I introduce an additional `select` constraint? Like "at most `n` frames"? I think I could multiply *something* by `lt(n)`, but I'm not sure I understand the `n=$k:e` syntax. Looking into it right now. – Atcold Apr 03 '17 at 15:31
  • Oh, getting it. `n` number of outputs set to `$k`, stream `:` evaluation `e` of `mod(n\,$k)+1`. – Atcold Apr 03 '17 at 15:40