I have a work project currently where my company is using a 3D printer, a couple of servo motors, and a raspberry pi to produce a robotic arm. This robotic arm is meant to be visually-guided. I am task to research and code the software for the robot to be able to pick up a specific object and place it at a specific location. I need some help on how I could program the robot to pick up the object using visual guidance.
I've made a set-up where the top-down image will very likely look like the picture below. The objective is to pick up a die (those that you can find in a casino or in a game of monopoly) inside a clearly marked area of operation and place it on a piece of paper with an 'X' marked on it.
This is what I've accomplished thus far:
- I am able to use OpenCV and Machine Learning to identify the die and the paper marked with 'X' and extract the coordinates in the image where the die is located (right now my camera is not located directly above the area of operations)
- I am able to determine the orientation of the die (i.e. 30 degrees clockwise, 42 degrees anti-clockwise, etc) - this is important because the robotic arm uses a caliper-like clipper to pick up the die and I would not want it to hold the die insecurely by the corner
- I am also able to code a program to move the robotic arm (something like servo1 to turn 30 degrees clockwise, servo2 to turn 40 degrees anti-clockwise, etc)
However, I have no idea how I could code the robotic arm to pick up the die. The die and the X in the photo are for illustration purposes only, their location could change to any spot inside the area of operation.
What is really depressing is I can't come up with any strategy. Hopefully, someone can advise on some principles and recommend some strategies - I'm not asking for code because I am confident to code my own software. I also don't think the programming language is important at this point but if it helps, I am using Python for this project - are there any libraries for such a task already? I felt like I have searched the entire web but haven't found any helpful tutorials on this yet.
Also, if it's helpful - I come from a web-software developer background usually coding in HTML, CSS, and Javascript. Python was the first language I've managed to master to a competent degree before I started coding with web technologies. I have some experience in C during my high school but have not coded in C for more than 10 years already.
Thanks for any help.
