5

We're working on a SpriteKit game. In order to have more control over sound effects, we switched from using SKAudioNodes to having some AVAudioPlayers. While everything seems to be working well in terms of game play, frame rate, and sounds, we're seeing occasional error(?) messages in the console output when testing on physical devices:

... [general] __CFRunLoopModeFindSourceForMachPort returned NULL for mode 'kCFRunLoopDefaultMode' livePort: #####

It doesn't seem to really cause any harm when it happens (no sound glitches or hiccups in frame rate or anything), but not understanding exactly what the message means and why it's happening is making us nervous.

Details:

The game is all standard SpriteKit, all events driven by SKActions, nothing unusual there.

The uses of AVFoundation stuff are the following. Initialization of app sounds:

class Sounds {
  let soundQueue: DispatchQueue

  init() {
    do {
      try AVAudioSession.sharedInstance().setActive(true)
    } catch {
      print(error.localizedDescription)
    }
    soundQueue = DispatchQueue.global(qos: .background)
  }

  func execute(_ soundActions: @escaping () -> Void) {
    soundQueue.async(execute: soundActions)
  }
}

Creating various sound effect players:

guard let player = try? AVAudioPlayer(contentsOf: url) else {
  fatalError("Unable to instantiate AVAudioPlayer")
}
player.prepareToPlay()

Playing a sound effect:

let pan = stereoBalance(...)
sounds.execute {
  if player.pan != pan {
    player.pan = pan
  }
  player.play()
}

The AVAudioPlayers are all for short sound effects with no looping, and they get reused. We create about 25 players total, including multiple players for certain effects when they can repeat in quick succession. For a particular effect, we rotate through the players for that effect in a fixed sequence. We have verified that whenever a player is triggered, its isPlaying is false, so we're not trying to invoke play on something that's already playing.

The message isn't that often. Over the course of a 5-10 minute game with possibly thousands of sound effects, we see the message maybe 5-10 times.

The message seems to occur most commonly when a bunch of sound effects are being played in quick succession, but it doesn't feel like it's 100% correlated with that.

Not using the dispatch queue (i.e., having sounds.execute just call soundActions() directly) doesn't fix the issue (though that does cause the game to lag significantly). Changing the dispatch queue to some of the other priorities like .utility also doesn't affect the issue.

Making sounds.execute just return immediately (i.e., don't actually call the closure at all, so there's no play()) does eliminate the messages.

We did find the source code that's producing the message at this link:

https://github.com/apple/swift-corelibs-foundation/blob/master/CoreFoundation/RunLoop.subproj/CFRunLoop.c

but we don't understand it except at an abstract level, and are not sure how run loops are involved in the AVFoundation stuff.

Lots of googling has turned up nothing helpful. And as I indicated, it doesn't seem to be causing noticeable problems at all. It would be nice to know why it's happening though, and either how to fix it or to have certainty that it won't ever be an issue.

bg2b
  • 1,939
  • 2
  • 11
  • 18
  • Why would you switch? `AvAudioPlayer` is built into `SKAudioNode` and you can access what you need from the `AVAudioNode`. Anyway, it looks like somebody is trying to grab from the audio port, but the port is not available at the time it is being grabbed. Perhaps it is because you are doing it on a background thread. – Knight0fDragon Oct 21 '19 at 18:59
  • Our understanding of the relationship of all the AV... classes isn't very comprehensive. The SKAudioNode approach worked fine for simple sounds, but when we wanted to have some stereo effects based on what was happening in the game, we just couldn't get SpriteKit to cooperate. Either we'd disable positional and the stereo wasn't working, or we'd enable positional and then there was fading and other undesired effects. If there's a way to get to an underlying AVAudioPlayer from an SKAudioNode's avAudioNode, we could try that. – bg2b Oct 21 '19 at 19:52
  • Oh, as mentioned we did try calling play on the main thread instead of a background thread (no async, just play()) to see what would happen. That's bad for the game's frame rate, but the message still occurs. – bg2b Oct 21 '19 at 19:55
  • weird that the audio would cause lag.... but anyway, cast the `avAudioNode` to an `AVAudioPlayerNode`, it should have everything you need in it. – Knight0fDragon Oct 21 '19 at 20:09
  • I tried adding an SKAudioNode, grabbing avAudioNode! as! AVAudioPlayerNode, setting pan, and running the node with SKAction.play(). It does play the sound, but I just cannot get it to pan. (Same as with the SKAction.stereoPan actions that we tried before.) Perhaps I'll try making some AVAudioPlayerNodes myself (instead of making AVAudioPlayers) and hooking them to the scene's audioEngine so that I have more knowledge of what's going on. – bg2b Oct 21 '19 at 23:14
  • It has been a while since I did this, but you cant create SKAudioNodes until after the scene is moved to the view. Did you do this? There is a way to determine if the audio node exists – Knight0fDragon Oct 21 '19 at 23:17
  • I used an action run 5 seconds after didMove(to:view). The audio node plays, and the cast audio.avAudioNode! as! AVAudioPlayerNode does work. If I make the audio node positional, put a listener at the center of the screen, and move the audio node to the left or right, I can hear the balance change, but the positional audio stuff mutes it significantly then. This was what we observed before. I'm going to try either hooking into or maybe disabling in some way the scene's audioEngine tomorrow morning. Perhaps there's some sort of interference between that and our AVAudioPlayers. – bg2b Oct 22 '19 at 01:15
  • FWIW, just shutting down the scene's audioEngine doesn't fix the issue. I'll try working with the engine, but I have to learn a fair amount and rework some stuff so it'll take a bit of time. Thanks for the comments; they've helped me to see more of the picture of how AV... works together for audio stuff, so at least I have some ideas for how to proceed. – bg2b Oct 22 '19 at 08:36

1 Answers1

3

We're still working on this, but have experimented enough that it's clear how we should do things. Outline:

Use the scene's audioEngine property.

For each sound effect, make an AVAudioFile for reading the audio's URL from the bundle. Read it into an AVAudioPCMBuffer. Stick the buffers into a dictionary that's indexed by sound effect.

Make a bunch of AVAudioPlayerNodes. Attach() them to the audioEngine. Connect(playerNode, to: audioEngine.mainMixerNode). At the moment we're creating these dynamically, searching through our current list of player nodes to find one that's not playing and making a new one if there's none available. That's probably got more overhead than is needed, since we have to have callbacks to observe when the player node finishes whatever it's playing and set it back to a stopped state. We'll try switching to just a fixed maximum number of active sound effects and rotating through the players in order.

To play a sound effect, grab the buffer for the effect, find a non-busy playerNode, and do playerNode.scheduleBuffer(buffer, ...). And playerNode.play() if it's not currently playing.

I may update this with some more detailed code once we have things fully converted and cleaned up. We still have a couple of long-running AVAudioPlayers that we haven't switched to use AVAudioPlayerNode going through the mixer. But anyway, pumping the vast majority of sound effects through the scheme above has eliminated the error message, and it needs far less stuff sitting around since there's no duplication of the sound effects in-memory like we had before. There's a tiny bit of lag, but we haven't even tried putting some stuff on a background thread yet, and maybe not having to search for and constantly start/stop players would even eliminate it without having to worry about that.

Since switching to this approach, we've had no more runloop complaints.

Edit: Some example code...

import SpriteKit
import AVFoundation

enum SoundEffect: String, CaseIterable {
  case playerExplosion = "player_explosion"
  // lots more

  var url: URL {
    guard let url = Bundle.main.url(forResource: self.rawValue, withExtension: "wav") else {
      fatalError("Sound effect file \(self.rawValue) missing")
    }
    return url
  }

  func audioBuffer() -> AVAudioPCMBuffer {
    guard let file = try? AVAudioFile(forReading: self.url) else {
      fatalError("Unable to instantiate AVAudioFile")
    }
    guard let buffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: AVAudioFrameCount(file.length)) else {
      fatalError("Unable to instantiate AVAudioPCMBuffer")
    }
    do {
      try file.read(into: buffer)
    } catch {
      fatalError("Unable to read audio file into buffer, \(error.localizedDescription)")
    }
    return buffer
  }
}

class Sounds {
  var audioBuffers = [SoundEffect: AVAudioPCMBuffer]()
  // more stuff

  init() {
    for effect in SoundEffect.allCases {
      preload(effect)
    }
  }

  func preload(_ sound: SoundEffect) {
    audioBuffers[sound] = sound.audioBuffer()
  }

  func cachedAudioBuffer(_ sound: SoundEffect) -> AVAudioPCMBuffer {
    guard let buffer = audioBuffers[sound] else {
      fatalError("Audio buffer for \(sound.rawValue) was not preloaded")
    }
    return buffer
  }
}

class Globals {
  // Sounds loaded once and shared amount all scenes in the game
  static let sounds = Sounds()
}

class SceneAudio {
  let stereoEffectsFrame: CGRect
  let audioEngine: AVAudioEngine
  var playerNodes = [AVAudioPlayerNode]()
  var nextPlayerNode = 0
  // more stuff

  init(stereoEffectsFrame: CGRect, audioEngine: AVAudioEngine) {
    self.stereoEffectsFrame = stereoEffectsFrame
    self.audioEngine = audioEngine
    do {
      try audioEngine.start()
      let buffer = Globals.sounds.cachedAudioBuffer(.playerExplosion)
      // We got up to about 10 simultaneous sounds when really pushing the game
      for _ in 0 ..< 10 {
        let playerNode = AVAudioPlayerNode()
        playerNodes.append(playerNode)
        audioEngine.attach(playerNode)
        audioEngine.connect(playerNode, to: audioEngine.mainMixerNode, format: buffer.format)
        playerNode.play()
      }
    } catch {
      logging("Cannot start audio engine, \(error.localizedDescription)")
    }
  }

  func soundEffect(_ sound: SoundEffect, at position: CGPoint = .zero) {
    guard audioEngine.isRunning else { return }
    let buffer = Globals.sounds.cachedAudioBuffer(sound)
    let playerNode = playerNodes[nextPlayerNode]
    nextPlayerNode = (nextPlayerNode + 1) % playerNodes.count
    playerNode.pan = stereoBalance(position)
    playerNode.scheduleBuffer(buffer)
  }

  func stereoBalance(_ position: CGPoint) -> Float {
    guard stereoEffectsFrame.width != 0 else { return 0 }
    guard position.x <= stereoEffectsFrame.maxX else { return 1 }
    guard position.x >= stereoEffectsFrame.minX else { return -1 }
    return Float((position.x - stereoEffectsFrame.midX) / (0.5 * stereoEffectsFrame.width))
  }
}

class GameScene: SKScene {
  var audio: SceneAudio!
  // lots more stuff

  // somewhere in initialization
  // gameFrame is the area where action takes place and which
  // determines panning for stereo sound effects
  audio = SceneAudio(stereoEffectsFrame: gameFrame, audioEngine: audioEngine)

  func destroyPlayer(_ player: SKSpriteNode) {
    audio.soundEffect(.playerExplosion, at: player.position)
    // more stuff
  }
}
bg2b
  • 1,939
  • 2
  • 11
  • 18