How to create voiceovers with the synthesizer and Musickit?

Question

My SwiftUI app is a workout app that uses the voice synthesizer to voice instructions over music played from local files using AVAudioPlayer. That player lets me lower the music volume so users can hear the voice.

Many users want to play Apple Music rather than my supplied music. I agree; great idea. I actually got playing playlists working with Musickit, but there appears to be no way for me to lower the music volume programmatically. So both the voice and the music play at the same volume, which doesn't work.

Is there a way to programmatically change the volume of the ApplicationMusicPlayer? It appears since Apple Music used DRM, there's no other audio player I can use, but I'm not sure.

Or is there a better way to accomplish what I'm trying to do?

thanks

Show your code, please. Everything depends on the AVAudioSession, but you haven't shown us what it is. — matt, Aug 15 '23 at 16:41
You need to [edit] your question to include all relevant code in the form of a [mcve] in order to make the question on-topic. — Dávid Pásztor, Aug 15 '23 at 16:42
There is no code. The only apparent way to play music, using Musickit, because of DRM, is using some form of MusicPlayer, which doesn't support setting the volume. As far as I know, which is why I'm asking. — Todd Hoff, Aug 15 '23 at 16:55

score 0 · Answer 1 · answered Aug 19 '23 at 00:04

Instead of reflexively asking for code when someone simply doesn't know how to do something, it's more helpful to tell them how to do it.

For example, there's something called duckmode that lowers the sound of music in the background. I did not know this, so there was no code.

It lowers music played using Musickit played in the same program, but it does not lower music played by AvAudioPlayer.

Before playing a sentence through the synthesizer, I call setDuckMode(), and when the sentence is complete, I set the active audio session back to mix mode.

This works, except there are timing problems when you want to play apple music, speak through the synth, and play a sound effect through AvAudioPlay all at the same time, which is something I need to do.

You can't switch modes while AvAudioPlayer is still playing audio, so you can't go into duck to hear the voice over Apple music. Trying to do so causes an error and interrupts the audio being played through AvAudioPlayer.

You can set a session to be in both duck and mix mode at the same time, but that keeps the music low the entire time.

Ideally, you want to schedule speaking through the synthesizer when audio is not being played through AvAudioPlayer.

func setDuckMode() {
              
                    do {
                        
                        try AVAudioSession.sharedInstance().setActive(false)
                        
                        try AVAudioSession.sharedInstance().setCategory(
                            .playback,
                            mode: .default,
                            options: [.duckOthers])
                   
                        try AVAudioSession.sharedInstance().setActive(true)
                        
                    } catch {
                        Logger.sm.debug("Could not init sound duck. \(error)")
                    }
}
    
func setMixMode() {
            
            do { 
                try AVAudioSession.sharedInstance().setActive(false)
                
                try AVAudioSession.sharedInstance().setCategory(
                    .playback,
                    mode: .default,
                    options: .mixWithOthers)
                
                try AVAudioSession.sharedInstance().setActive(true)
               
            } catch {
                Logger.sm.debug("Could not init sound mixed. \(error)")
            }
 }
        
 extension SoundManager:  AVSpeechSynthesizerDelegate {
            
                func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) {
          
                    setMixMode()
                }
                
}

How to create voiceovers with the synthesizer and Musickit?

1 Answers1