I'm having problems playing audio in DialogFlow(api.ai) with SSML from webhook written in python

Question

In my python webhook, I am returning JSON response as-

   {
     "speech":"Speak this",
     "ssml": <speak><audio src="[link to a mp3 file]"></audio></speak>,
     "displayText": "Expected text",
     "contextOut": [],
     "source": ""
   }

However, what I get in DialogFlow from "Show JSON" is-

  "fulfillment": {
      "speech": "Speak this",
      "source": "",
      "displayText": "Expected text",
      "messages": [
        {
          "type": 0,
          "speech": "Speak this"
        }
      ]
    }

Here, the response got no SSML key. Also, if I remove the "speech" key from my program, I get this error in Google assistant test app, MalformedResponse expected_inputs[0].input_prompt.rich_initial_prompt.items[0].simple_response: 'text_to_speech' or 'ssml' must be set.

Please help!

ANSWER

"speech":'<speak><audio src="[link to a mp3 file]"></audio></speak>'

Did the trick

Have you tried making the speech field the ssml? – Nick Felker Mar 10 '18 at 19:27 — Nick Felker, Mar 10 '18 at 19:27
Yes, I figured that out, anyway thanks @NickFelker – S4rt-H4K Mar 10 '18 at 21:33 — S4rt-H4K, Mar 10 '18 at 21:33

score 0 · Answer 1 · answered Mar 10 '18 at 23:31

As I think you might have realized in your answer, there are two issues

The field name is "speech", not "ssml".
You weren't providing a JSON string in your reply.

Real JSON requires that the string be a double-quote, not a single quote (although it seems Dialogflow's parser is being lenient), so it should have been:

   {
     "speech": "<speak><audio src='[link to a mp3 file]'></audio></speak>",
     "displayText": "Expected text",
     "contextOut": [],
     "source": ""
   }

I'm having problems playing audio in DialogFlow(api.ai) with SSML from webhook written in python

1 Answers1