2

MS recently introduced direct speech channel and some samples for web frontend to use it. But i was wondering is it a good fit for use in call center scenario using some SIP or services like twilio phone? If so i would like to see some docs how to use direct line speech api and wire it up to some telephony? I've already created github issue but it stay wo attention https://github.com/MicrosoftDocs/bot-docs/issues/1162

PS: also i have related problem, i can't find any docs on how to exachange secret to direct line token. Link for original direct line is not working for speech direct line. Thanks

Andrey Stepanov
  • 311
  • 2
  • 11

2 Answers2

5

Andrey! Thanks for your interest. I'm Travis, and I work with the engineering teams building Direct Line Speech.

You're correct that the concepts behind Direct Line Speech are a great fit for call center scenarios. There's a huge potential value to application there. At this stage, in preview, we don't currently have built-in support for SIP/RTP to seamlessly integrate telephony I/O, but this is a need that we've heard loud and clear and we're actively investigating solutions as we move forward.

The good news, though, is that you're not blocked on getting started--you can make this work today, just with a little more legwork. Conceptually, you can use an existing telephony endpoint solution (like what Ram's pointing to) as a "middle tier" between your telephony clients and your Direct Line Speech bot. This middle tier service, which could itself actually be a simple bot, would be responsible for handling the RTP/SIP and then forwarding audio between the end user and the "real" bot, using the Speech SDK there to connect with Direct Line Speech. This is a little clunky, to be sure, but so long as the services are co-located within regions, it should still be able to produce a high-quality, low-latency experience the same way you'd get out of a native client.

Thanks again, and please keep sharing your thoughts and questions on the products; we're using your feedback to actively guide where we focus and what we create.

Travis Wilson
  • 111
  • 2
  • Thanks i will surely investigate this solutions. But frankly now i'm a bit confused how to even start with direct line speech. 1) I have taken speech example as most close to my needs https://github.com/Microsoft/BotFramework-WebChat/tree/master/samples/13.customization-speech-ui but i can't find where i can exchange secret into token, link with standart direct line not working. 2) i'm not sure what exactly should i pass data after enabling streaming endpoint (text after tts, or just audio, data or stream) and where should i pass it. – Andrey Stepanov Jun 08 '19 at 17:44
  • @travis How does one pass audio stream directly to DLS endpoint? https://stackoverflow.com/questions/58212779/how-to-pass-media-stream-to-the-direct-line-speech-endpoint – whihathac Oct 03 '19 at 05:33
1

Please find the docs section that has Tutorials and so-on. we have the Direct Line Speech channel with which a few lines of code to the above assistant enables you to stream audio to one endpoint and benefit from STT/Bot/TTS all in one call – audio is streamed back.

The steps to add Speech are here. For CallCenter scenarios to integrate with the telephony system (PBX) of the customer, Speech Bot / VIL could take calls (SIP and RTP) from their PBX. Please find the docs for Voice first virtual assistants.

Ram
  • 2,459
  • 1
  • 7
  • 14
  • i've investigated all that links already. Most interesting for me was last link, but that's also generic article on , i was heading to the link https://github.com/Azure-Samples/cognitive-services-speech-sdk from there, but looks like it also contain just some STT, TTS examples and what i want is to see simple how-to to connect to SIP. You mentioned that bot "could take calls" from PBX, but how to configure that, which endpoints and payloads shoould i use? – Andrey Stepanov May 29 '19 at 19:50
  • AudioCodes’ Voice.AI Gateway brings the most intuitive form of human communications to your chatbot service, supporting phone and WebRTC voice calls. Please refer the voice.AI gateway link(https://www.audiocodes.com/solutions-products/solutions/audiocodes-voiceai/voiceai-gateway). – Ram May 30 '19 at 06:00
  • seems a bit like advertisement, but ok, can you please lead me to some code samples, 'cos site is bit complicated. – Andrey Stepanov Jun 02 '19 at 21:22
  • The above sample code not available publicly. Alternatively You can explore the SIP trunk provider that you use to integrate with Bot from the following list Nexmo, NetFoundry, Twilio. The following can be worth to explore Sample Code for Realtime Transcription using Nexmo, Microsoft Azure Speech Services & websockets(https://github.com/nexmo-community/voice-microsoft-speechtotext). – Ram Jun 06 '19 at 14:29