0

I'm using ESP32-Cam WebServer and capture the cam stream in python OpenCV+YOLOv8.
It works, but there is significant lag.
The delay is approximately 2 seconds after what the camera captures.

Here's my code:

import cv2
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
cap = cv2.VideoCapture("http://ip")


while cap.isOpened():
    success,frame = cap.read()


    if success:
        results = model(frame)

        annotated_frame = results[0].plot()

        cv2.imshow("YOLOv8 Inference",annotated_frame)


        if cv2.waitKey(1) & 0xFF ==ord("q"):
            break
    else:
        break
    
cap.release()
cv2.destroyAllWindows()

Is this due to insufficient hardware performance of the ESP32 or are there any modifications that can be made to the code?

ryanhuang1124
  • 185
  • 1
  • 1
  • 9

2 Answers2

2

The AI processing slows everything down.

This causes video buffers to inflate, which causes apparent latency.

Drop the AI processing as a test. You'll probably see it play back with less delay.

To fix this, run the AI processing in its own thread, and make sure you don't force it to process every frame.

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
0

Streaming itself is a bit of an art. That delay is probably created by some buffering of the streaming app. You should look at the documentation of the stream up to see how you can delay the latency. Normally there's a tradeoff between smoothness, latency and quality.

E. Soria
  • 71
  • 6