1

I am trying to find examples on how to use getUserMedia stream object to createPushStream with the Azure Speech SDK. Note that I intend to run the code in Safari, so the use of MediaRecorder is not possible. The intent is to use getUserMedia stream to feed both the SpeechSDK.SpeechRecognizer (SpeechSDK.AudioConfig.fromStreamInput) and to save stream as an audio file. SpeechSDK.AudioConfig.fromDefaultMicrophoneInput does not allow that.

MadeInLagny
  • 185
  • 1
  • 12

1 Answers1

0

One way to do this would be to grab the audio data sent to the service via the connection.messageSent callback. Build the recognizer in the usual way (you should be able to use fromDefaultMicrophoneInput), then add this callback:

    const con: sdk.Connection = sdk.Connection.fromRecognizer(r);

    let wavFragmentCount: number = 0;

    const wavFragments: { [id: number]: ArrayBuffer; } = {};

    con.messageSent = (args: sdk.ConnectionMessageEventArgs): void => {
        if (args.message.path === "audio" && args.message.isBinaryMessage && args.message.binaryMessage !== null) {
            wavFragments[wavFragmentCount++] = args.message.binaryMessage;
        }
    };

Once recognition is completed, wavFragments has the audio data captured, which you can write to a file.

(This example code was taken from ConnectionTest.ts )

glenn
  • 36
  • 4
  • Thanks ! However, I am not using node but javascript in the browser. So, to get wav file, I need to convert it to a blob and create a download link. I tried const blob = new Blob([this.wavFragments]) but it does not create a valid wav file. Any idea on how to get a blob out of these wavFragments ? – MadeInLagny Oct 14 '20 at 08:55
  • I can now answer my own question: const blob = new Blob([sentAudio]) does create a valid blob. – MadeInLagny Oct 14 '20 at 10:48