1

I am running a pose tracking application on Colab (with mediapipe). It does not show a continuous video, instead it runs my video frame by frame, showing them in succession in the output section below my block of code. So it becomes very slow at processing a single video and I have the output section full of frames, so I have to scroll a lot of stuff to check the top or the bottom of the output section. The goal is to have a video stream like a normal linux application on my PC.

This is the main() file of my application

cap = cv2.VideoCapture('1500_doha.mp4')
pTime = 0
detector = poseDetector()
while cap.isOpened():

    success, img = cap.read()
    width,height, c=img.shape
    img = detector.findPose(img)
    lmList = detector.findPosition(img, draw=False)
    angle=detector.findAngle(img, 11, 13, 15) #attenzione, cambia braccio ogni tanto!!
    cTime = time.time()
    fps = 1 / (cTime - pTime)
    pTime = cTime
    text=cv2.putText(img, str(int(fps)), (70, 50), cv2.FONT_HERSHEY_PLAIN, 3,
                (255, 0, 0), 3)
    cv2_imshow(img)
    cv2.waitKey(10)

The problem is clearly in cv2_imshow(), because if I run a YOLO-V4 box detector I don't need this command and I obtain a continuous stream. Have you any suggestions? Is there already a solution online?

Here you find part of the output box of my google colab.

Here you find the complete file https://colab.research.google.com/drive/1uEWiCGh8XY5DwalAzIe0PpzYkvDNtXID#scrollTo=HPF2oi7ydpdV

desertnaut
  • 57,590
  • 26
  • 140
  • 166
furio_19
  • 11
  • 2

1 Answers1

0

The function cv_imshow() is quite new by the time. So for ages you were forced to use another library (e.g. matplotlib's imshow) or simply write a file and open it later.

As far as I know, by now there is no patch for video processing via OpenCV, similar to cv2_show() since it is very slow, so if you truly want to see the video in the inline, you will have to use other methods.

One I found out is using the function clear_output from IPython.display, so you are constantly clearing and writing. I warn you, it is very slow and it usually crashes, I'd rather visualize it writing and opening it afterward.

Here is my code.

import cv2
from google.colab.patches import cv2_imshow
from IPython.display import clear_output

# Create a VideoCapture object and read from input file
# If the input is the camera, pass 0 instead of the video file name
cap = cv2.VideoCapture(path+'Video.mp4')
 
# Check if camera opened successfully
if (cap.isOpened()== False): 
  print("Error opening video stream or file")
 
# Default resolutions of the frame are obtained.The default resolutions are system dependent.
# We convert the resolutions from float to integer.
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
 
# Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file.
out = cv2.VideoWriter('output.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
 
# Read until video is completed
while(cap.isOpened()):
  # Capture frame-by-frame
  ret, frame = cap.read()
  if ret == True:
    ##########

    # YOUR PROCESSING HERE



    ##########
 
    # Write the frame into the file 'output.avi'
    out.write(frame)
    clear_output(wait=True)
    # Display the resulting frame
    cv2_imshow(frame) # This is not the openCV function, it would not work outside Google colab

    # Press Q on keyboard to  exit
    if cv2.waitKey(1) & 0xFF == ord('q'):
      break
 
  # Break the loop
  else: 
    break
 
# When everything done, release the video capture object
cap.release()
 
# Closes all the frames
cv2.destroyAllWindows()
Alex Barnadas
  • 1
  • 1