4

I want to make a web-app which does video calls with live transcription -- using some 3rd party speech-to-text service (e.g. Google or Amazon). So the peer-to-peer MediaStream would be played to the users, and also sent to the API for transcription.

I am currently using https://peerjs.com/ to create the peer-to-peer call.

Is this feasible? Are there any code examples or libraries I could use?

Thank you, Daniel

Daniel Winterstein
  • 2,418
  • 1
  • 29
  • 41
  • Something like [this](https://cloud.google.com/speech-to-text/docs/streaming-recognize#speech-streaming-recognize-nodejs) but for web? – quicVO May 20 '21 at 19:27

1 Answers1

3

I think it can easily be done with Azure speach to text service, Perhaps other solutions may be used, but for Azure I was able to quickly find all pieces.

There is browser use case example Here

This line is responsible for getting audio source from microphone, but in you case more interesting would be to use: fromStreamInput function, which accepts MediaStream.

I’m also wondering what would be better place to execute this process:

  • on source machine, send audio to receiver and to speachSDK, and after getting transcription send it afterwards.
  • on receiver machine, after receiving use SDK to get transcription, in this case audio may have worse quality as it was compressed during transmission, therefore you may get worse transcription.

However this can be easily be tested.

mjpolak
  • 721
  • 6
  • 24
  • 1
    That looks great - except I think it runs server-side. The browser example has a required server hosting part. – Daniel Winterstein May 27 '21 at 08:27
  • 1
    This `php` file in the example is misleading, see requirements: `Optionally, a web server that supports hosting PHP scripts.` It is used only for generating token used to access SpeachSDK, see this: https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/js/browser/index.html#L260 It is not wise to have such token on browser side, someone would be able to steal it and use service on your bill. – mjpolak May 27 '21 at 08:32