playback raw pcm from network using AudioQueue in CoreAudio

Question

I need to play raw PCM data (16 bit signed) using CoreAudio on OS X. I get it from network using UDP socket (on sender side data is captured from microphone).
The problem is that all I hear now is some short cracking noise and then only silence.
I'm trying to play data using AudioQueue. I setup it like this:

// Set up stream format fields
AudioStreamBasicDescription streamFormat;
streamFormat.mSampleRate = 44100;
streamFormat.mFormatID = kAudioFormatLinearPCM;
streamFormat.mFormatFlags = kLinearPCMFormatFlagIsBigEndian | kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked;
streamFormat.mBitsPerChannel = 16;
streamFormat.mChannelsPerFrame = 1;
streamFormat.mBytesPerPacket = 2 * streamFormat.mChannelsPerFrame;
streamFormat.mBytesPerFrame = 2 * streamFormat.mChannelsPerFrame;
streamFormat.mFramesPerPacket = 1;
streamFormat.mReserved = 0;

OSStatus err = noErr;
// create the audio queue
err = AudioQueueNewOutput(&streamFormat, MyAudioQueueOutputCallback, myData, NULL, NULL, 0, &myData->audioQueue);
if (err)
{ PRINTERROR("AudioQueueNewOutput"); myData->failed = true; result = false;}

// allocate audio queue buffers
for (unsigned int i = 0; i < kNumAQBufs; ++i) {
    err = AudioQueueAllocateBuffer(myData->audioQueue, kAQBufSize, &myData->audioQueueBuffer[i]);
    if (err)
    { PRINTERROR("AudioQueueAllocateBuffer"); myData->failed = true; break; result = false;}
}

// listen for kAudioQueueProperty_IsRunning
err = AudioQueueAddPropertyListener(myData->audioQueue, kAudioQueueProperty_IsRunning, MyAudioQueueIsRunningCallback, myData);
if (err)
{ PRINTERROR("AudioQueueAddPropertyListener"); myData->failed = true; result = false;}

MyAudioQueueOutputCallback is:

void MyAudioQueueOutputCallback(void* inClientData,
                            AudioQueueRef inAQ,
                            AudioQueueBufferRef inBuffer)
{
    // this is called by the audio queue when it has finished decoding our data.
    // The buffer is now free to be reused.
    MyData* myData = (MyData*)inClientData;

    unsigned int bufIndex = MyFindQueueBuffer(myData, inBuffer);

    // signal waiting thread that the buffer is free.
    pthread_mutex_lock(&myData->mutex);
    myData->inuse[bufIndex] = false;
    pthread_cond_signal(&myData->cond);
    pthread_mutex_unlock(&myData->mutex);
}

MyAudioQueueIsRunningCallback is:

void MyAudioQueueIsRunningCallback(void* inClientData,
                               AudioQueueRef inAQ,
                               AudioQueuePropertyID inID)
{
    MyData* myData = (MyData*)inClientData;

    UInt32 running;
    UInt32 size;
    OSStatus err = AudioQueueGetProperty(inAQ, kAudioQueueProperty_IsRunning, &running, &size);
    if (err) { PRINTERROR("get kAudioQueueProperty_IsRunning"); return; }
    if (!running) {
        pthread_mutex_lock(&myData->mutex);
        pthread_cond_signal(&myData->done);
        pthread_mutex_unlock(&myData->mutex);
    }
}

and MyData is:

struct MyData
{
AudioQueueRef audioQueue;                                                           // the audio queue
AudioQueueBufferRef audioQueueBuffer[kNumAQBufs];           // audio queue buffers

AudioStreamPacketDescription packetDescs[kAQMaxPacketDescs];        // packet descriptions for enqueuing audio

unsigned int fillBufferIndex;       // the index of the audioQueueBuffer that is being filled
size_t bytesFilled;                         // how many bytes have been filled
size_t packetsFilled;                       // how many packets have been filled

bool inuse[kNumAQBufs];                     // flags to indicate that a buffer is still in use
bool started;                                       // flag to indicate that the queue has been started
bool failed;                                        // flag to indicate an error occurred
bool finished;                                      // flag to inidicate that termination is requested

pthread_mutex_t mutex;                      // a mutex to protect the inuse flags
pthread_mutex_t mutex2;         // a mutex to protect the AudioQueue buffer
pthread_cond_t cond;                        // a condition varable for handling the inuse flags
pthread_cond_t done;                        // a condition varable for handling the inuse flags
};

I'm sorry if I posted too much code - hope it helps anyone to understand what exactly I do.

Mostly my code based on this code which is version of AudioFileStreamExample from Mac Developer Library adapted to work with CBR data.

Also I looked at this post and tried AudioStreamBasicDescription desribed there. And tried to change my flags to Little or Big Endian. It didn't work.
I looked at some another posts here and in the other resources while finding similar problem, I checked the order of my PCM data, for example. I just can't post more than two links.

Please anyone help me to understand what I'm doing wrong! Maybe I should abandon this way and use Audio Units right away? I'm just very newbie in CoreAudio and hoped that mid-level of CoreAudio will help me to solve this problem.

P.S. Sorry for my English, I tried as I can.

Answer 1

I hope you've solved this one on your own already, but for the benefit of other people who are having this problem, I'll post up an answer.

The problem is most likely because once an Audio Queue is started, time continues moving forward, even if you stop enqueueing buffers. But when you enqueue a buffer, it is enqueued with a timestamp that is right after the previously enqueued buffer. This means that if you don't stay ahead of the where the audio queue is playing, you will end up enqueuing buffers with a timestamp in the past, therefore the audio queue will go silent and the isRunning property will still be true.

To work around this, you have a couple of options. The simplest in theory would be to never fall behind on submitting buffers. But since you are using UDP, there is no guarantee that you will always have data to submit.

Another option is that you can keep track of what sample you should be playing and submit an empty buffer of silence whenever you need to have a gap. This option works good if your source data has timestamps that you can can use to calculate how much silence you need. But ideally, you wouldn't need to do this.

Instead you should be calculating the timestamp for the buffer using the system time. Instead of AudioQueueEnqueueBuffer, you'll need to use AudioQueueEnqueueBufferWithParameters instead. You just need to make sure the timestamp is ahead of where the queue is currently at. You'll also have to keep track what the system time was when you started the queue, so you can calculate the correct timestamp for each buffer you are submitting. If you have timestamp values on your source data, you should be able to use them to calculate the buffer timestamps as well.