13

I'm looking for a Python video processing library, similar to PIL, where I can iterate through all the frames of a source video, access the pixel data for each frame, draw onto each frame and save the result as a new video file.

I've found a couple of similar questions, but they are pretty old now:

They recommend PyMedia and PyFFMPEG. PyMedia seems rather out of date (but may still work?) and PyFFMPEG, while more recent, has almost no documentation.

I've had no luck installing either these on Ubuntu 10.10, before I press on, is there:

a) A better library I should look at?

b) Good instructions on how to get either of these up and running?

Community
  • 1
  • 1
Tom
  • 42,844
  • 35
  • 95
  • 101

5 Answers5

9

I've often needed the same thing and as far as I know, there is no good solution with bindings in Python.

Also it is not as simple as it may seem to manipulate frames of a video file. A modern file format for video does not store the frames one frame after the other but instead uses "delta frames", in which only the changes from one frame to the other is stored. Other considerations such as video with variable frame rate makes the problem even harder.

In the past I've used the following command to generate images from video.

ffmpeg -i /path/to/file.mpg -an -r 30 -s 320x240 tmp%06d.jpg

Where 30 is the target frame rate, 320x240 the image dimension and tmp%06d.jpg the pattern to use to store the generated jpegs. Then you can use PIL to manipulate each frame and mencoder or ffmpeg to stich the images back again into a movie:

ffmpeg -r 30 -i tmp%06d.jpg output.mpg

Obviously, you'll lose the audio track.

juliomalegria
  • 24,229
  • 14
  • 73
  • 89
Björn Lindqvist
  • 19,221
  • 20
  • 87
  • 122
  • Yeah, that's what I've been trying but it seems pretty wasteful and like there should be a solution which cuts out the intermediate image files (as there are 10000s of them). I understand the issue with the key frames, but I'm not expecting a lossless process here. The audio track can be stripped out and mixed back in easily enough with mencoder, and presumably ffmpeg too. – Tom Apr 11 '11 at 10:41
  • This is a reasonable starter approach and maybe an ok simplification, but it's probably too much of a reduction to say 'modern file format for video.. uses delta frames.' The algorithms vary quite a lot by codec and many different types of frames are used in some of the more complex codecs, and most don't actually keep all the data (lossy vs lossless). For example, H264 will try to keep the frame data that matters for human perception. – Profane Sep 12 '11 at 12:57
3

I recommend scikit-video for you which is the most straightforward python video processing library I have ever met.

This is the official introduction to scikit-video from its project website:

Scikit-video is designed for easy video processing using Python. It is modeled in the spirit of other successful scikits such as scikit-learn and scikit-image. The developers of scikit-video know libraries exist for manipulating videos, such as PyFFmpeg, MoviePy, PyAV, imageIO, and opencv. However, no libraries have been found to provide an all-in-one solution for research-level video processing tools.

Beanocean
  • 133
  • 10
2

I've been looking at using py.processing for similar work. It does everything you're asking, but is a hybrid with Processing. You're not writing python code per se. Anyway, It makes it quite easy to work with, but there is a lot of program/interpretation overhead so it can be slow to do realtime modification of the movies. You said you wanted to edit to file so that should be workable.

Poltergeister
  • 21
  • 2
1

You can use my VidGear video-processing python library's WriteGear API that allows us to exploit almost all available parameters supported by FFmpeg (framerate, bitrate, codecs, format, and size,mux, demux etc.) in Compression Mode, effortlessly and flexibly and while doing that it robustly handles errors/warnings all along very quietly.

Parameters:

For example, To use H.264 for producing a high-quality video using the encoder x264, we can tweak its parameters as following to produce lossless output video:

 output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast", "tune": "zerolatency"}

and then pass this dictionary to WriteGear as the example given below

Basic Usage Example

# import libraries
from vidgear.gears import WriteGear
import cv2

output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer

stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device

writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4' 

# infinite loop
while True:

    (grabbed, frame) = stream.read()
    # read frames

    # check if frame empty
    if not grabbed:
        #if True break the infinite loop
        break


    # {do something with frame here}
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # write a modified frame to writer
        writer.write(gray) 

        # Show output window
    cv2.imshow("Output Frame", frame)

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

cv2.destroyAllWindows()
# close output window

stream.release()
# safely close video stream
writer.close()
# safely close writer

Checkout more advance usage details here and complete docs here

abhiTronix
  • 1,248
  • 13
  • 17
0

I find imageio the easiest to use video manipulation library for Python. Makes reading videos frame by frame, their processing and saving back to disk a breeze. Nowadays the authors also provide an easy to pip install wrapper for ffmpeg that can be used in Windows, Linux or OSX. There is also an active community here answering questions related to the library under the [python-imageio] tag.

Mr K.
  • 1,064
  • 3
  • 19
  • 22