iPhone AudioStreamer AudioQueue Realtime FFT

Question

I'm using Matt Gallagher's AudioStreamer to play a mp3 audio stream. Now I want to do FFT in realtime and visualize the frequencies using OpenGL ES on the iPhone.

I'm wondering where to catch the audio data and pass it to my "Super-Fancy-FFT-Computing-3D-Visualization-Method". Matt is using the AudioQueue Framework and there is a Callback function that is set with:

err = AudioQueueNewOutput(&asbd, ASAudioQueueOutputCallback, self, NULL, NULL, 0, &audioQueue);

The Callback looks like this:

static void ASAudioQueueOutputCallback(void*                inClientData, 
                                AudioQueueRef           inAQ, 
                                AudioQueueBufferRef     inBuffer){...}

In the moment I'm passing the data from the AudioQueueBufferRef and the result looks very weird. But with FFT and visualizations there are so many points where you can screw it up that I wanted to be sure to pass at least the right data to the FFT. I'm reading the data from the Buffer this way ignoring every second value because I only want to analyze one channel:

SInt32* buffPointer = (SInt32*)inBuffer->mAudioData;
int count = 0;
for (int i = 0; i < inBuffer->mAudioDataByteSize/2; i++) {
    myBuffer[i] = buffPointer[count];
    count += 2;
}

Then follows FFT computing with myBuffer containing 512 values.

I just found out that it is not possible to just catch the PCM data at some point using the code from Matt's AudioStreamer. The data is passed to the AudioQueue in the compressed format. But I'm going to continue my research, there must be a way... — benjamin.ludwig, Oct 04 '12 at 08:52

score 3 · Accepted Answer · answered Oct 04 '12 at 18:12

Instead of sending the data you receive from the audio file stream callback directly to the audio queue, you could convert it to PCM, run your analysis, and then feed it to the audio queue (as PCM) if you still need to play it. To do the conversion, you could use Audio Converter Services (which will be a screaming nightmare without end), or an offline audio queue.

Option 3: look into the new Audio Queue "tap" on iOS 6, which lets you look at data inside a queue. I still need to check this out… it looks cool (and I'm giving a talk on it three weeks at CocoaConf, so, yeah…)

(repost from: http://lists.apple.com/archives/coreaudio-api/2012/Oct/msg00034.html )

The Audio Queue "tap" was the absolute hint! It's not yet part of the iOS 6 doc but you can find everything you need in the header file AudioQueue.h. Just look for AudioQueueProcessingTap. — benjamin.ludwig, Oct 04 '12 at 18:22

iPhone AudioStreamer AudioQueue Realtime FFT

1 Answers1