12

I would like to build an Android App to take audio data from two microphones, mix the sound with some from memory, and play the sound through headphones. This needs to be done in real-time. Could you please refer me to some tutorials or any references, for real-time audio input, mixing, and output with Java eclipse?

So far, I am able to record sound, save it, and then play it, but I cannot find any tutorials for real-time interfacing with sound-hardware this way.

Note: One microphone is connected to the 3.5 mm headphone jack of the Android through a splitter and the other is connected through a USB port.

Thanks!

user1854928
  • 123
  • 1
  • 5
  • 2
    You realize that it can never be in `real-time` because it needs to go through the wires and then some processing, right? – kush Nov 26 '12 at 23:29
  • 2
    At this point, I'm willing to take up to a few seconds of delay. – user1854928 Nov 29 '12 at 00:40
  • I am sorry. I wish I could help but this is beyond me as well. All I can do is upvote. By the way, you are using `android.media.AudioRecord` and not `android.media.MediaRecorder`, right? – kush Nov 29 '12 at 10:49
  • @kush, you have a curious understanding of "real-time". By that logic, nothing is real-time. Also light takes quite some time to travel to your eyes. – Prof. Falken Dec 13 '12 at 10:39
  • If the Android device has stereo microphone in, you could connect two mono microphones to it, instead of one USB. – Prof. Falken Dec 13 '12 at 10:41

1 Answers1

5

There are two issues that I see here:

1) Audio input via USB.
Audio input can be done using android 3.2+ and libusb but it is not easy (You will need to get the USB descriptors from libusb, parse them yourself and send the right control transfers to the device etc). You can get input latency via USB in the order of 5-10 mS with some phones.

2) Audio out in real-time. This is a perennial problem in Android and you are pretty much limited to the Galaxy Nexus at the moment if you want to approach real-time (using Native Audio output). However, if you master the USB you may be able to output with less latency as well.

I suppose if you go to the trouble of getting the USB to work, you can get a USB audio device with stereo in. If you had connected one mono mic to each of the input channels, then output via USB you would be very close to your stated goal. You might like to try "USB Audio Tester" or "usbEffects" apps to see what is currently possible.

In terms of coding the mixing and output etc, you will probably want one thread reading each separate input source and writing to a queue in small chunks (100-1000 samples at a time). Then have a separate thread reading off the queue(s) and mixing, placing the output onto another queue and finally a thread (possibly in native code if not doing output via USB) to read the mixed queue and do output.

The following Link http://code.google.com/p/loopmixer/ has a flavor for dealing with the audio itself.

hack_on
  • 2,532
  • 4
  • 26
  • 30
  • Any idea how to implement the usb audio device input? The AudioRecord class seems limited for choosing input. – Gusdor Feb 14 '13 at 09:48
  • 1
    @Gusdor There is no built in Android API support for it that I know of. You have to use libusb and call this through JNI. The hard parts are parsing all the descriptors and timing the packets. – hack_on Feb 14 '13 at 20:40