0

I'm trying to record a video (640 x 360 MJPEG) with this camera using python and opencv but i can't get to the camera rated FPS.

This is the snippet I'm currently using:

stream = cv2.VideoCapture(1, cv2.CAP_DSHOW)
stream.set(cv2.CAP_PROP_FRAME_HEIGHT, 360)
stream.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
stream.set(cv2.CAP_PROP_FORMAT, cv2.CAP_OPENCV_MJPEG)
t0 = cv2.getTickCount()
while (cv2.getTickCount() - t0)/cv2.getTickFrequency() < 10:
    ret, frame = stream.read()
    frames.append(frame)
total = (cv2.getTickCount()-t0)/cv2.getTickFrequency()
print("Total time:", total, "# of Frames:", len(frames), "fps:", len(frames)/total)

The output I get is Total time: 10.0027773 # of Frames: 1874 fps: 187.34796784888934

The camera is rated for 260fps but I can't really get close to that. If I try to print the elapsed time of each frame this is what i get for the first few frames:

1 - 0.3112283
2 - 0.3267393, before+0.015511
3 - 0.3289934, before+0.0022541
4 - 0.3311813, before+0.0021879
5 - 0.3427716, before+0.0115903

As you can see there are two main problems:

  • it's not starting from zero
  • once every 3 frames the read function takes horribly longer (this goes on for the whole bunch of frames)

Any clue about why and how to overcome this? Thanks in advance

Ro.oT
  • 623
  • 6
  • 15
Ruka
  • 1
  • 2
  • Your [mcve] is incomplete, so we can't tell what size frames you are capturing or what format. – Mark Setchell Aug 08 '23 at 07:56
  • The frame size is 640 x 360 in MJPEG format – Ruka Aug 08 '23 at 10:38
  • could you update your code with the part where you configure the camera for that resolution and format? – Sembei Norimaki Aug 08 '23 at 10:56
  • actually I don't need to set them because they are already set from camera side, but I've added them. – Ruka Aug 08 '23 at 12:19
  • And what if you do `print(frame.shape)` after each `stream.read()`? – Mark Setchell Aug 08 '23 at 12:20
  • this is the result at each cycle: (360, 640, 3) – Ruka Aug 08 '23 at 12:33
  • That's RGB (because it ends in 3) rather than MJPEG. I don't think you have initialised it correctly to MJPEG and the interface probably can't handle the extra bandwidth caused by it not being compressed to JPEG so you won't get the speed. Try checking with `stream.get(cv2.CAP_PROP_FORMAT)` – Mark Setchell Aug 08 '23 at 12:43
  • It answers with -1.0, also, the `stream.set(cv2.CAP_PROP_FORMAT, cv2.CAP_OPENCV_MJPEG)` returns False – Ruka Aug 08 '23 at 12:51
  • opencv does color space conversion. that costs a bit every time. there is a cap prop to get raw data, for some backends. – Christoph Rackwitz Aug 08 '23 at 16:00
  • MJPEG FOURCC does still give you BGR/3channel data (opencv conversion to BGR). the MJPEG is compression on the usb wire. MJPEG does not go with FORMAT, it goes with FOURCC. – Christoph Rackwitz Aug 08 '23 at 16:05
  • I tried with `stream.set(cv2.CAP_PROP_FOURCC, cv2.CAP_OPENCV_MJPEG)`, it returns False (so I guess the change did't succeed), but the read function returns `False, None` at each iteration – Ruka Aug 09 '23 at 07:12
  • wrong value too. needs to be `VideoWriter_fourcc(*"MJPG")` and the order of set() calls matters too. -- and this still won't guarantee the frame rate you want. opencv still converts to BGR (from whatever yuv those cameras usually emit), costing some time, unless you disable the CONVERT_RGB prop. you may want the BGR conversion anyway. it may not even matter if it's done within the time it takes for the next frame to arrive. – Christoph Rackwitz Aug 09 '23 at 07:54
  • I tried `stream.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"MJPG")`, it returned True, and now i'm stuck at 30fps, even if I restart the camera or unplug and replug it in. Also it changed the format to 640x480. If I open it with AMCap the resolution and imposed Framerate is correct but as soon as I try to use it with OpenCV it gets stuck at 30FPS. – Ruka Aug 09 '23 at 07:56
  • enumerate your camera's video modes. I think ffmpeg/ffprobe can do that. ffmpeg wiki has recipes and magic spells for that. you'll need to be sure to request exactly those parameters for exactly that capture mode. and, still, order of set() calls matters. it's silly but who's gonna rewrite opencv's videoio? right. – Christoph Rackwitz Aug 09 '23 at 07:57
  • I'm sorry I'm not getting what exactly I should do – Ruka Aug 09 '23 at 08:00
  • while I compile a magic spell from this, please browse it: https://trac.ffmpeg.org/wiki/DirectShow – Christoph Rackwitz Aug 09 '23 at 08:11
  • I've installed ffmpeg and run `ffmpeg -f dshow -list_options true -i video="HD USB Camera"`, this is the result: `[[dshow @ 00000188a7529900] vcodec=mjpeg min s=640x360 fps=260.004 max s=640x360 fps=260.004 (pc, bt470bg/bt709/unknown, center) [dshow @ 00000188a7529900] vcodec=mjpeg min s=1280x720 fps=120 max s=1280x720 fps=120 (pc, bt470bg/bt709/unknown, center) [dshow @ 00000188a7529900] vcodec=mjpeg min s=1920x1080 fps=60.0002 max s=1920x1080 fps=60.0002 (pc, bt470bg/bt709/unknown, center)` – Ruka Aug 09 '23 at 10:11
  • opencv should manage to request any of these three modes. again, you have to play around with the order of the set() calls. that usually matters. – Christoph Rackwitz Aug 09 '23 at 10:31
  • oh also there's CAP_PROP_FPS, set/get, – Christoph Rackwitz Aug 09 '23 at 10:46
  • I tried this: `print(stream.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"MJPG"))) print(stream.set(cv2.CAP_PROP_FRAME_HEIGHT, 360)) print(stream.set(cv2.CAP_PROP_FRAME_WIDTH, 640)) print(stream.set(cv2.CAP_PROP_FPS, 260))` in any order you can think of, each of them returns True, but if i check the params with the get right after the set it does not take effect, apart for the fps one that changes also the frame height from 480 to 360 – Ruka Aug 09 '23 at 10:56
  • hmmmm fascinating. I wish I had a device to reproduce this with. can I assume that you're using the latest opencv, 4.8.0? `cv2.__version__` – Christoph Rackwitz Aug 09 '23 at 11:02
  • Yes, I was using 4.7.0, but even after the update the bahaviour didn't change – Ruka Aug 09 '23 at 12:04
  • you could try CAP_MSMF instead of dshow. that's also a windows API, newer. opencv's interfaces for dshow and msmf behave differently in some situations. -- you could try capturing with ffmpeg, giving those settings, and sending the data into the void (`ffmpeg ... -f null -`). {ffmpeg can do it} == {issue lies with ffmpeg} that wiki page ought to have a recipe for requesting those parameters. they're different from the usual ffmpeg filter stuff. they're parameters given to the input module in ffmpeg that handles dshow. – Christoph Rackwitz Aug 09 '23 at 12:09
  • With CAP_MSMF I get the same behaviour apart from not being able to set the fourcc (returns False) – Ruka Aug 09 '23 at 14:05

0 Answers0