Streaming PCM audio over tcp socket

Question

I have a continuous stream of raw PCM audio data from a TCP socket and I want to play them. I've done so many researches and saw many samples but no result. This gist was the most close solution but the problem is, it's streaming mp3 file. So I have a socket which receives linear PCM audio data and give them to the player like this:

func play(_ data: Data) {
    // this function is called for every 320 bytes of linear PCM data.
    // play the 320 bytes of PCM data here!
}

So is there any "Simple" way to play raw PCM audio data?

hotpaw2 · Answer 1 · 2020-01-06T15:31:29.857

2

For iOS, you can use the RemoteIO Audio Unit or the AVAudioEngine with a circular buffer for real-time audio streaming.

You can't give network data directly to audio output, but instead should put it in a circular buffer from which an audio subsystem play callback can consume it at its fixed rate. You will need to pre-buffer some amount of audio samples to cover network jitter.

Simple "ways" of doing this might not handle network jitter gracefully.

edited Jan 06 '20 at 15:31

answered Jan 06 '20 at 15:26

hotpaw2

70,107
14
90
153

Thanks. I'll search these keywords. Do you know any library or sample code doing similar job? – Sepehr Behroozi Jan 07 '20 at 07:27

score 2 · Answer 2 · answered Apr 08 '20 at 09:37

Answering late but If you are still stuck in playing TCP bytes then try to follow my answer where you put your tcp audio bytes in Circular Buffer and play it via AudioUnit.   Below code receives bytes from TCP and put them into a TPCircularBuffer  

func tcpReceive() {
        receivingQueue.async {
            repeat {
                do {
                    let datagram = try self.tcpClient?.receive()
                    var byteData = datagram?["data"] as? Data
                    let dataLength = datagram?["length"] as? Int


                    let _ = TPCircularBufferProduceBytes(&self.circularBuffer, byteData.bytes, UInt32(decodedLength * 2))

                } catch {
                    fatalError(error.localizedDescription)
                }
            } while true
        }
    }

Create Audio Unit  ...

var desc = AudioComponentDescription(
            componentType: OSType(kAudioUnitType_Output),
            componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
            componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
            componentFlags: 0,
            componentFlagsMask: 0
        )

        let inputComponent = AudioComponentFindNext(nil, &desc)

        status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
        if status != noErr {
            print("Audio component instance new error \(status!)")
        }

 // Enable IO for playback
        status = AudioUnitSetProperty(
            audioUnit!,
            kAudioOutputUnitProperty_EnableIO,
            kAudioUnitScope_Output,
            kOutputBus,
            &flag,
            SizeOf32(flag)
        )
        if status != noErr {
            print("Enable IO for playback error \(status!)")
        }

//Use your own format, I have sample rate of 16000 and pcm 16 Bit
        var ioFormat = CAStreamBasicDescription(
            sampleRate: 16000.0,
            numChannels: 1,
            pcmf: .int16,
            isInterleaved: false
        )     

//This is playbackCallback      var playbackCallback = AURenderCallbackStruct(
            inputProc: AudioController_PlaybackCallback, //This is a delegate where audioUnit puts the bytes
            inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
        )

        status = AudioUnitSetProperty(
            audioUnit!,
            AudioUnitPropertyID(kAudioUnitProperty_SetRenderCallback),
            AudioUnitScope(kAudioUnitScope_Input),
            kOutputBus,
            &playbackCallback,
            MemoryLayout<AURenderCallbackStruct>.size.ui
        )
        if status != noErr {
            print("Failed to set recording render callback \(status!)")
        }
//Init Audio Unit
  status = AudioUnitInitialize(audioUnit!)
        if status != noErr {
            print("Failed to initialize audio unit \(status!)")
        }

//Start AudioUnit      status = AudioOutputUnitStart(audioUnit!)
        if status != noErr {
            print("Failed to initialize output unit \(status!)")
        }

This is my playbackCallback function where I play the audio from Circular Buffer  

func performPlayback(
        _ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>,
        inTimeStamp: UnsafePointer<AudioTimeStamp>,
        inBufNumber: UInt32,
        inNumberFrames: UInt32,
        ioData: UnsafeMutablePointer<AudioBufferList>
    ) -> OSStatus {
        let buffer = ioData[0].mBuffers

        let bytesToCopy = ioData[0].mBuffers.mDataByteSize
        var bufferTail: UnsafeMutableRawPointer?

        var availableBytes: UInt32 = 0
        bufferTail = TPCircularBufferTail(&self.circularBuffer, &availableBytes)
        let bytesToWrite = min(bytesToCopy, availableBytes)


        var bufferList = AudioBufferList(
            mNumberBuffers: 1,
            mBuffers: ioData[0].mBuffers)

        var monoSamples = [Int16]()
        let ptr = bufferList.mBuffers.mData?.assumingMemoryBound(to: Int16.self)
        monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(inNumberFrames)))
        print(monoSamples)

        memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
        TPCircularBufferConsume(&self.circularBuffer, bytesToWrite)

        return noErr
    }

For TPCircularBuffer I used this pod

'TPCircularBuffer', '~> 1.6'

score 0 · Answer 3 · answered Jan 07 '20 at 08:41

All detail description and sample code is available for

Audiotoolbox / AudioUnit

You can register the Callback to get the PCM data from the AUGraph and send the pcm buffer to the socket.

Some more example of the usage :

https://github.com/rweichler/coreaudio-examples/blob/master/CH08_AUGraphInput/main.cpp

Streaming PCM audio over tcp socket

3 Answers3