How to send command to arduino when a bounding box is detected?

Question

i am working on a robot that detects users and follows. I am using YOLOv2 from darkflow repo for the object detection module.

As of now, i want to make the Arduino micro-controller to move the robot forward towards the bounding box when detected. There is no distance or stereo cam. It is just if bounding box detected then drive the motors forward.

This is my code for detecting bounding box in through the camera. It will be helpful if someone could direct me to the right path for any resources or tutorials. Thank you.

import cv2
from darkflow.net.build import TFNet
import numpy as np
import time

options = {
    'model': 'cfg/yolov2.cfg',
    'load': 'bin/yolov2.weights',
    'threshold': 0.8,
    'gpu': 0.8
}

tfnet = TFNet(options)
colors = [tuple(255 * np.random.rand(3)) for _ in range(10)]

capture = cv2.VideoCapture(0)
capture.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)

while True:
    stime = time.time()
    ret, frame = capture.read()
    if ret:
        results = tfnet.return_predict(frame)
        for color, result in zip(colors, results):
            tl = (result['topleft']['x'], result['topleft']['y'])
            br = (result['bottomright']['x'], result['bottomright']['y'])
            label = result['label']
            confidence = result['confidence']
            text = '{}: {:.0f}%'.format(label, confidence * 100)
            frame = cv2.rectangle(frame, tl, br, color, 5)
            frame = cv2.putText(frame, text, tl, cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 0), 2)
        cv2.imshow('frame', frame)
        print('FPS {:.1f}'.format(1 / (time.time() - stime)))
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

capture.release()
cv2.destroyAllWindows()

When there is a person detected, there will be bounding box around the person. I want to move the robot towards the person when detected or bounding box is formed. As i can think of, the only way will be trying to get the center position of (x,y) of the bounding box and send the command to arduino through serial comm. It will be helpful if i can get the center position of the bounding box. — , Dec 16 '19 at 00:14
This sounds like a design issue, no? In any case, it still seems too broad for Stack Overflow. — AMC, Dec 16 '19 at 00:16

Piglet · Accepted Answer · 2019-12-16T08:46:42.873

1

It will be helpful if i can get the center position of the bounding box

As you have the top left and bottom right corner of the bounding box the center is the average of both positions.

How to send commands to arduino?

Use serial communication through the virtual COM port of your Arduino. There are literally a million tutorials online.

https://www.arduino.cc/reference/en/language/functions/communication/serial/

https://pyserial.readthedocs.io/en/latest/

https://www.instructables.com/id/Interface-Python-and-Arduino-with-pySerial/

edited Dec 16 '19 at 08:46

answered Dec 16 '19 at 08:40

Piglet

27,501
3
20
43

Thank you! I will read those and come back if there is any confusion. – Dec 16 '19 at 09:24

How to send command to arduino when a bounding box is detected?

1 Answers1