opencv - video looks good but frames are rotated 90 degrees

Question

We save a video on a mobile client and send it to a server. On the server, I use the following code to save the frames:>

import skvideo.io
import cv2

haar = 
'/home/ubuntu/opencv/data/haarcascades/haarcascade_frontalface_alt.xml'
face_cascade = cv2.CascadeClassifier(haar)

ret = True

video = 'my_video.mov'
i = 0
while ret == True:
    cap = skvideo.io.VideoCapture(video)
    ret, frame = cap.read()
    cv2.imwrite('frame_'+str(i)+'.jpg',frame)
    i+=1

When we play the video on a windows media player or itunes, it looks good. I.e. the player knows how to orient it.

But skvideo.io does not know that, and those frames we saved are rotated 90 degrees counterclockwise.

How can we embed info into the video file (a .mov) file that that skvideo knows the correct orientation?

Any reason for using `skvideo.io` over the `cv2.VideoCapture` API ? — ZdaR, Jun 06 '17 at 04:42

score 7 · Accepted Answer · answered Jun 09 '17 at 12:18

7

there was a glitch in skvideo, it was not reading the available metadata. For videos taken in mobile are rotated, but metadata includes such parameter. The skvideo team committed a fix, and current skvideo version 1.1.7 reads metadata from mobile, that indicates that video should be rorated. skvideo.io.vread then rotates the file:

1) use newer skvideo version, 1.1.7 which can be cloned at https://github.com/scikit-video/scikit-video

2) You can use following code to read all frames in the video, most likely metadata will be read

import skvideo.io
videogen = skvideo.io.vread(f.name)

That will rotate the video automatically if it was taken in portrait mode.

3) Created an issue on skvideo repo, take a look for further reference: https://github.com/scikit-video/scikit-video/issues/40

answered Jun 09 '17 at 12:18

Alejandro Simkievich

3,512
4
33
49

thanks! If you upvote my solution, I will not be offended! :) – Alejandro Simkievich Jun 09 '17 at 13:08
Confirmed this is still an issue even with OpenCV 4... Using scikit-video works for me, thanks to its integration with ffmpeg to read the video orientation / rotation metadata. – Daniel Sposito Apr 13 '20 at 00:06
apparently solved in opencv 4.5 https://github.com/opencv/opencv/issues/15499 – Ta946 Feb 24 '21 at 09:54

score 4 · Answer 2 · answered Jun 06 '17 at 08:40

It looks like OpenCV does not record the rotation metadata of the video file with VideoCapture() as you can see by the propIds that it stores.

I'm not sure if scikit-video does. It looks like they have a metadata puller called ffprobe which might be able to pull the rotation. See here for an example of how to call and see the output. This shows a hefty list of metadata---no rotation---but that might just be because it's not set or of a movie type which doesn't have rotation metadata.

Another way to grab it would be to read the metadata directly from ffmpeg. I found an old StackOverflow answer that wrote a little python code to extract specifically the rotation metadata from a video using ffmpeg.

@AlejandroSimkievich if you do find a precise solution with `scikit-video` or `ffmpeg` you should edit your original question with it, so that others with this question can easily get to something that works! — alkasm, Jun 07 '17 at 01:48

score 1 · Answer 3 · edited Apr 06 '21 at 10:44

I installed 'ffmpeg' ('ffprobe'). My computer is Ubuntu 18.04. Then Python, get_rotation() function worked for me.

Function Code:

import subprocess
import shlex
import json
def get_rotation(self, file_path_with_file_name):
    cmd = "ffprobe -loglevel error -select_streams v:0 -show_entries stream_tags=rotate -of default=nw=1:nk=1"
    args = shlex.split(cmd)
    args.append(file_path_with_file_name)
    ffprobe_output = subprocess.check_output(args).decode('utf-8')
    if len(ffprobe_output) > 0:  # Output of cmdis None if it should be 0
        ffprobe_output = json.loads(ffprobe_output)
        rotation = ffprobe_output
    else:
        rotation = 0
    return rotation

opencv - video looks good but frames are rotated 90 degrees

3 Answers3

Linked