Streaming audio data Using Audio Queue Services drops audio after first few buffers

Question

I've been trying to get audio data to be played when it's being received but from the current implementation I found had flaws. I've modified it somewhat, but there are almost no examples of this case of streaming live audio data. The issue occurs when playing the beginning of the streamed data. It will play a few buffers, then when I monitor what is going on with the queue's being called, the AudioOutputCallback method goes rogue. It will call properly for the first few times then call more than the amount of buffers I have allocated (about 6 or more times). Currently I've tried manually calling the callback method and it will call but not cycle through the 3 buffers allocated.

Some specifications I have are, the data is coming in at 1000 Bytes and it never came in, in anything other than that, so it's pretty constant. Audio Queue is only used for Playback of streamed data from the network. 3 buffers are allocated as Apple suggested there be. They are somewhat reused. This implementation is supposed to be some sort of circular buffer.

Here is my current implementation:

Initialization:

public func setupAudioQueuePlayback() {
    incomingData = NSMutableData()
    audioQueueBuffers = []
    isBuffersUsed = []
    var outputStreamDescription = createLPCMDescription()
    AudioQueueNewOutputWithDispatchQueue(&audioPlaybackQueue, &outputStreamDescription, 0, playbackQueue, AudioQueueOutputCallback)
    createAudioBuffers()
    AudioQueueStart(audioPlaybackQueue!, nil)
}

CreateBuffers method:

fileprivate func createAudioBuffers() {
    for _ in 0..<allocatedBuffersUponStart {
        var audioBuffer: AudioQueueBufferRef? = nil
        let osStatus = AudioQueueAllocateBuffer(audioPlaybackQueue!, UInt32(1000), &audioBuffer)
        if osStatus != noErr {
            print("Error allocating buffers!"); return
        } else {
            self.audioQueueBuffers.append(audioBuffer)
            self.isBuffersUsed.append(false)
            AudioQueueAllocateBuffer(audioPlaybackQueue!, UInt32(1000), &audioBuffer)
        }
    }
}

Where the data is entering method:

private func playFromData(_ data: Data) {
    playbackGroup.enter()
    playbackQueue.sync {
        incomingData.append(data)
        var bufferIndex = 0
        while true {
            if !isBuffersUsed[bufferIndex] {
                isBuffersUsed[bufferIndex] = true
                break
            } else {
                bufferIndex += 1
                if bufferIndex >= allocatedBuffersUponStart {
                    bufferIndex = 0
                }
            }
        }
        currentIndexGlobal = bufferIndex
        let bufferReference = audioQueueBuffers[bufferIndex]
        bufferReference?.pointee.mAudioDataByteSize = UInt32(incomingData.length)
        bufferReference?.pointee.mAudioData.advanced(by: 0).copyMemory(from: incomingData.bytes, byteCount: incomingData.length)
        AudioQueueEnqueueBuffer(audioPlaybackQueue!, bufferReference!, 0, nil)
        incomingData = NSMutableData()
        playbackGroup.leave()
    }
}

Callback Method (Not Global):

private func AudioQueueOutputCallback(aq: AudioQueueRef, buffer: AudioQueueBufferRef) {
    for index in 0..<allocatedBuffersUponStart {
        if isBuffersUsed[index] == true {
            isBuffersUsed[index] = false
        }
    }
}

If your wondering how it's all being implemented:

func inSomeMethodOutsideThisFile() {
    audioService = AudioService.shared
    audioService.setupAudioQueuePlayback()
    dataManager.subscribe(toTopic: "\(deviceId)/LineOutAudio", qoS: .messageDeliveryAttemptedAtLeastOnce) { (audioData) in
        self.audioService.playFromData(audioData)
    }
}

I've tried other ways but this way was the main way it all started.