-2

It was successful to train from jet son-XAVIER and recognize it as cam. But I don't know how to send the information that jet son-XAVIER recognized in real time to Arduino. The objects we recognize are elk and wild boar. I know that I can only send 1 letter through communication, so elk will send e and wild boar to w. Is there a way to send it as soon as it recognizes it through real-time web cam?

GWJ
  • 1

2 Answers2

0

There's not a lot of information here on your setup, but here's a possible solution:

I see the NVIDIA Jetson AGX Xavier module has USB-C ports. Buy a USB-A to USB-C cable, and plug the Arduino directly in.

I'm not sure what program/language you're using with your trained model, but I'll guess that it's python for now.

You'll want to open a Serial connection to the arduino, and you can do this with pyserial: https://pypi.org/project/pyserial/

You can send more than just one letter, you can send entire data streams. But, if you want to just send one letter, it will do that as well.

Here's the official documentation for how to communicate with an Arduino using Python: https://create.arduino.cc/projecthub/ansh2919/serial-communication-between-python-and-arduino-e7cce0

If you're not using python, specify your language of choice, and we can look up and see if it has a serial library.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Bucky
  • 76
  • 5
  • Thank you for your kind reply. To further explain the problem we are experiencing now, we are running webcam using the weight file after training with yolov4. './darknet detector demo data/obj.data data/yolo-obj.cfg backup/yolo-obj_last.weights' was used to execute the weight file learned to see elk and wild boar appear on the real-time webcam. I want to send the values of elk and wild boar on web cam to arduino in this real time. – GWJ Jul 14 '21 at 11:39
  • First of all, I'm looking for a way to do it without using ros. I thought of adding a phrase to darknet.py as a way, but I don't know if it's the right way, and if it's the right way, I don't know what phrase to add. The person who answers can let me know if you know any other ways. Thank you again for your kind reply. – GWJ Jul 14 '21 at 11:39
  • @GWJ What is the Jetson running for an OS right now? – Bucky Jul 14 '21 at 15:43
  • I`m using unbuntu(18.04) on jetson. And I`m using jetpack 4.5 which NVDIA provides. Jetpack 4.5 contains opencv 4.1.1, cuda 10.2, cudnn8.0. :) – GWJ Jul 15 '21 at 08:08
  • @GWJ, when you say you're getting real time detections, is that from running a python script? How do you start your camera and get detections? At the point where you successfully print "found an elk" or "found a wild boar", on the Jetson, that is where you would use pySerial to send information to the Arduino. The example Ricardo gave shows you how this would work. – Bucky Jul 16 '21 at 19:16
0

I have never used darknet but may be this can point you in the right direction. I have used the library sugested by Bucky and I believe you could add the serial comunication to darknet.py. This is what I would do:

#Add this import at begining of the file darknet.py
import serial

#########################################################
#this is a mocked version of detect in darknet.py, assuming that the labels you used are "elk" and "wildboard". You should not add this lines to the file.
def detect():       
    res = []
    res.append(("elk",0.98,(2,2,50,50)))
    res.append(("wildboard",0.98,(2,2,50,50)))
    return res  

r = detect()
##########################################################

#Add this after the 'print r' at the end of the file darknet.py

ser = serial.Serial('/dev/ttyUSB0')  # open serial port. You should check what serial port is assigned to your arduino.

for obj in r:
    if obj[0]=="elk" and obj[1]>=0.9: #assuming 0.9 as the minimum confident for a detection
        print "found elk"
        ser.write('e') # you can send a string with just one letter 
    elif obj[0]=="wildboard" and obj[1]>=0.9:
        print "found wildboard";
        ser.write('w') # you can send a string with just one letter 

ser.close() # close port
Dharman
  • 30,962
  • 25
  • 85
  • 135
Ricardo Morales
  • 67
  • 4