1

i have an assignment on one of my classes. i need to "make" a musical instrument with a phone. I was thinking of making something like this:

there is a musical staff on a wall (or a board). then, the user can stick musical notes on that staff and point their phone camera to that wall. then the phone detects the musical notes (that could be crotchet or a half note, even a G cleff) and plays that succession of notes as sounds. the musical notes will have the same size and color because they will not be drawn, we will make them out of paper or cardboard.

my concern is if it's possible to detect several custom known shapes (in this case a bunch of musical notes), and their positioning regarding another one (in this case the lines on the staff, or the staff itself) using openCv; if not, any other idea of how could do this would be great.

maybe you could point me the direction on what to look for in google to get me started on this matter. i have 2 and a half months to do this project ( if it's relevant for you to know this ), so i would appreciate any help. thank you in advance.

angel208
  • 281
  • 1
  • 5
  • 15
  • Post a few images, and what you tried so far.. – Miki Oct 24 '15 at 19:37
  • @Miki i haven't started yet. i just want to know if its a viable project. i want to know if its possible to detect that kind of (equal) shapes because all i have found is how to detect different shapes with different colors, and i think that wouldn't help. what images do you need specifically? – angel208 Oct 24 '15 at 20:13
  • You can probably start from [here](http://sourceforge.net/projects/musicsheetrecognitionlibrary/), and see how they use OpenCV internally. – Miki Oct 25 '15 at 13:05

1 Answers1

1

What you're asking is a big picture idea of how this can be solved. Here are some thoughts to get you started.

First, you need to identify where the musical note in the image is. If you know the camera is always going to be static, you could do some background subtraction technique. A more rigorous approach would be to identify musical notes in the image.

Have a look at the MNIST dataset. It has a bunch of handwritten numbers and the corresponding ground truth. You could start here and extend this to musical notes. To begin, you can use k-nearest to classify these images.

Utkarsh Sinha
  • 3,295
  • 4
  • 30
  • 46
  • thank you, your answer tells me that at least its not a crazy-hard idea. i think the app doesn't need real-time detection. maybe the phone could take the picture of the staff and then process that picture to play the sounds. does this sounds right? – angel208 Oct 26 '15 at 12:10