How to improve performance net.forward() of cv2.dnn.readNetFromCaffe() , net.forward taking more time(7 to 10 seconds/frame) to give the result

Question

I have used net = cv2.dnn.readNetFromCaffe(protoFile, weightsFile) and then looping through the live video frames to get the outputs for each frames using net.forward().

But the net.forward() takes 7 to 10 seconds for each frames to give the result. Please help me how to improve the performance (reduce the time taking to process in net.forward()).

Means: From Step1 to Step2 takes 7 to 10 seconds for each frames.

(Step1 and Step2 are mentioned in the below code).

import cv2
import time
import numpy as np

protoFile = "deploy.prototxt"
weightsFile = "iter_10.caffemodel"

inWidth = 300
inHeight = 300

# web camera
cap = cv2.VideoCapture(0)
hasFrame, frame = cap.read()

net = cv2.dnn.readNetFromCaffe(protoFile, weightsFile)
k = 0
while 1:
    k+=1
    t = time.time()
    print("Start time = {}".format(t))
    hasFrame, frame = cap.read()

    if not hasFrame:
        cv2.waitKey()
        print("Wait====>")
        break

    inpBlob = cv2.dnn.blobFromImage(frame, 1.0 / 255, (inWidth, inHeight),
                              (0, 0, 0), swapRB=False, crop=False)


    net.setInput(inpBlob)

    # Step1
    print("before forward = {}".format(time.time() - t))

    output = net.forward()

    # Step2
    #taking close to 7 to 10 seconds for each frame
    print("forward = {}".format(time.time() - t))

@DmitryKurtaev , I used openpose pre-trained models and this model name is "pose_deploy.prototxt" and openpose stored their weight file as "pose_iter_102000.caffemodel". For sure this model is of "VGG" type. Please find their github link: https://github.com/CMU-Perceptual-Computing-Lab/openpose/tree/master/models/hand — Vasant, Feb 06 '19 at 22:34

score 2 · Answer 1 · answered Feb 08 '19 at 04:28

OpenPose models are huge by itself. There are several ways to improve an efficiency.

1) Try a trunk version (.caffemodel, .prototxt).

2) Lower input blob resolution. Note that it may reduce accuracy significantly.

3) Try a different model. You may take a look onto option to build OpenCV with Intel's Inference Engine (OpenVINO) and try this model: https://github.com/opencv/open_model_zoo/blob/2018/intel_models/human-pose-estimation-0001/description/human-pose-estimation-0001.md. The third option if only you have Intel CPU or GPU or Movidius Neural Compute Stick.

Could you give me some link how i can compress or trunk the .caffemodel and .prototxt file. I haven't seen any model available for Hand key-point identifications in OpenVINO . (As of now, I could see only openpose can have this trained model and weight file). — Vasant, Feb 14 '19 at 01:52

score 2 · Answer 2 · answered Feb 03 '20 at 17:51

I reduce the image size before passing it through the network.

# convert the input frame from BGR to RGB then resize it to have
# a width of 750px (to speed up processing)
rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
rgb = imutils.resize(frame, width=750)
imageBlob = cv2.dnn.blobFromImage(cv2.resize(rgb, (300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0))

# apply OpenCV's deep learning-based face detector to localize
# faces in the input image
detector.setInput(imageBlob)
detections = detector.forward()

How to improve performance net.forward() of cv2.dnn.readNetFromCaffe() , net.forward taking more time(7 to 10 seconds/frame) to give the result

2 Answers2

Linked