How to get video metadata from bytes using imageio.v3?

Question

I am creating a python class to process videos received from a http post. The videos can be from a wide range of sizes, 10 seconds up to 10 hours. I am looking for a way to get video metadata such as fps, height, width etc. without having to store the whole video in memory.

The class is initialized like:

class VideoToolkit:
   def __init__(self,video,video_name,media_type,full_video=True,frame_start=None,frame_stop=None):
    
    self._frames = iio.imiter(video,format_hint=''.join(['.',media_type.split('/')[1]])) # Generator
    self._meta = iio.immeta(video,exclude_applied=False)

The line of self._meta doesn't work giving an error:

OSError: Could not find a backend to open `<bytes>`` with iomode `r`.

Is there a similar way to get metadata using imageio.v3 and not storing the whole video in memory?

Just as an example, it is possible to get the metadata directly opening a video from a file:

import imageio.v3 as iio

metadata = iio.immeta('./project.mp4',exclude_applied=False)
print(metadata)

Output:

{'plugin': 'ffmpeg', 'nframes': inf, 'ffmpeg_version': '4.2.2-static https://johnvansickle.com/ffmpeg/ built with gcc 8 (Debian 8.3.0-6)', 'codec': 'mpeg4', 'pix_fmt': 'yuv420p', 'fps': 14.25, 'source_size': (500, 258), 'size': (500, 258), 'rotate': 0, 'duration': 1.69}

But opening the same file as bytes, this didn't work:

import imageio.v3 as iio
with open('./project.mp4', 'rb') as vfile:
    vbytes = vfile.read()
    metadata = iio.immeta(vbytes,exclude_applied=False)
    print(metadata)

Output:

OSError: Could not find a backend to open `<bytes>`` with iomode `r`.

PS: One way could be doing next(self._frames) to get the first frame and then get its shape, but the video fps would be still missing.

FirefoxMetzger · Accepted Answer · 2022-09-19T13:57:06.603

You are correct that you'd use iio.immeta for this. The reason this fails for you is because you are using the imageio-ffmpeg backend which makes the decision if it can/can't read something based on the ImageResource's extension. bytes have no extension, so the plugin will think it can't read the ImageResource. Here are different ways you can fix this:

import imageio.v3 as iio

# setup
frames = iio.imread("imageio:cockatoo.mp4")
video_bytes = iio.imwrite("<bytes>", frames, extension=".mp4")

# set the `extension` kwarg (check the docs I linked)
meta = iio.immeta(video_bytes, extension=".mp4")

# use the new-ish pyav plugin
# (`pip install av` and ImageIO will pick it up automatically)
meta = iio.immeta(video_bytes)

Note 1: Using pyav is actually preferable, because it extracts metadata without decoding pixels. This is faster than imageio-ffmpeg, which internally calls ffmpeg in a subprocess, will decode some pixels and then discard that data (expensive noop). This is especially true when reading from HTTP resources.

Note 2: In v2.21.2, the pyav plugin doesn't report FPS, only duration where availabe. There is now a PR (853) that adds this (and other things), but it will likely not get merged for the next few weeks, because I am busy with my PhD defense. (now merged)

Note 3: Many people interested in FPS want to know this info to calculate the total number of frames in the video. In this case, it can be much easier to call iio.improps and inspect the resulting .shape, e.g., iio.improps("imageio:cockatoo.mp4", plugin="pyav").shape # (280, 720, 1280, 3)

About the note 3, it is not my cause because I treat a video a video frame-by-frame and to reconstruct it later I need to know the FPS. But the solution you post works perfectly well. — brenodacosta, Sep 15 '22 at 14:53
@brenodacosta Yes, if you don't want to know the shape of the resulting data then this point doesn't apply to you. I've added it for others that might come across this answer, because in this case `improps` is preferable to `immeta`. — FirefoxMetzger, Sep 19 '22 at 13:58

How to get video metadata from bytes using imageio.v3?

1 Answers1