1

I am using openCv for making video processing. What I do is reading a video frame by frame, then applying some processing on each frame and then displaying the new modified frame. My code looks like that :

video_capture = cv2.VideoCapture('video.mp4')


while True:
# Capture frame-by-frame
  ret, frame = video_capture.read()

# Applying some processing to frame
                 .
                 .
                 .
# Displaying the new frame with processing
  img=cv2.imshow('title', frame)

  if cv2.waitKey(1) & 0xFF == ord('q'):
      break

This way I can display instantaneously the processed video. The problem is the display is lagging much due to the presence of the 'waitkey'. Is there another way for display images in real time to form a video, but with another module than cv2?

Thank you

Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
haffj
  • 11
  • 2
  • 4
  • What frame rate is the video? – Dan Mašek May 23 '17 at 17:32
  • 1
    Also, have you done any measurements to support your conclusion? I highly suspect that the frame processing dominates the time. That aside, I see further issues: a) no attempt at some consistent timing of playback (`VideoCapture` just reads/decodes the file) b) no buffering of processed frames c) processing and display in same thread. IMHO you should address those issues first. – Dan Mašek May 23 '17 at 19:23
  • Yes you are right most part of lagging is due to the processing. That said even without any processing, only reading the video with opencv slows down the frame rate. I measured 30fps for the original video, 17fps when displaying it with opencv and 6fps by adding processing. And waitkey is taking 50ms/frame. To speed up the processing I in parallelization – haffj May 24 '17 at 06:59
  • To speed up the processing I introduced parallelization (with the module multiprocessing), and it effectively reduces the time of processing. But when I use it on video, it seems that the waitkey interfers with the multiprocessing, and so the video with parallelization is even slower. – haffj May 24 '17 at 07:05
  • I'm pretty new to video processing but I guess the problem as you said can be because processing and display are in same thread. Please do you have some links to help me about the 3 points you mentioned? Thank you – haffj May 24 '17 at 07:09
  • @Dan Mašek Hi, please what do you mean by buffering of processed frames? I've been looking for some information, and the only thing I found is displaying the frames by using the linux frame buffer. Would it make any sense to do this in my case? Thank you – haffj Jun 06 '17 at 07:14
  • It's a case like [procude-consumer problem](https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem). You have one thread that reads frames from some source and fills a queue up to some size. Then there's an another thread, which is responsible for asynchronously displaying those frames at some rate. The queue in this case serves as a [data buffer](https://en.wikipedia.org/wiki/Data_buffer) -- if the reading slows down for some reason temporarily, there are enough frames to display.. – Dan Mašek Jun 06 '17 at 23:31

1 Answers1

0

One option is Tkinter, you can find some information here. Along with Tkinter, it uses python-gstreamer and python-gobject. It is much more complicated to set up, however it allows for more customization options.

Nate
  • 51
  • 7