Correct fulfillmentMessages format for DialogFlow Google Assistant

Question

I'm trying to use the Java DialogFlow API to build a webhook response for Google Assistant. My responses work fine in DialogFlow's 'Try It Now' feature, but Google Assistant keeps saying "isn't responding right now. Try again later".

As an experiment, I managed to get Google Assistant working by using a non-webhook response (i.e. through the normal DialogFlow Intent UI). When I looked in my History, I see the working response looked like this:

"queryText": "GOOGLE_ASSISTANT_WELCOME",
  "action": "input.welcome",
  "fulfillmentMessages": [
    {
      "text": {
    "text": [
      "[{\"type\":\"simple_response\",\"platform\":\"google\",\"textToSpeech\":\"Hello\"}]"
    ]
      }
    }

This seems very strange to me, as the text body is actually a further JSON encoded object (containing textToSpeech amongst other fields). When I use the Java DialogFlow API like this:

List<String> texts = new ArrayList<>();
texts.add( "Foo" );
message.setText( new GoogleCloudDialogflowV2IntentMessageText().setText( texts ));

I get a different format:

  "fulfillmentMessages": [
    {
      "text": {
    "text": [ "Foo" ]
      }
    }

That Google Assistant says:

MalformedResponse: Failed to parse Dialogflow response into AppResponse because of empty speech response"

Even if I try forcing an encoded JSON string as my text body, it still doesn't seem to work.

What is the correct way to return a default message format so that Google Assistant can read it? I tried simpleResponse and that didn't work either

sgodsell · Accepted Answer · 2019-05-16T23:26:30.060

1

The following is a minimum response needed to hear a response

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
            "simpleResponse": {
              "textToSpeech": "Welcome! Do you want me to change color or pause spinning? You can also tell me to ask you later."
            }
          }
        ]
      }
    }
  }
}

Naturally with java you can use the API's to generate the needed output responses. Plus if you use basicCards or Images for smart displays, then it definitely helps a lot more to use the API's instead. You can also check the response from the simulator.

edited May 16 '19 at 23:26

answered May 15 '19 at 15:55

sgodsell

26
4

Thanks for your suggestion! Can you please edit your response to show where this JSON goes from the 'top level'? Is it inside 'fulfillmentMessages'? – Richard Kennard May 15 '19 at 20:46
I tried your suggestion but got same problem. I have pasted the exact JSON here: https://stackoverflow.com/questions/56158748/dialogflow-why-does-this-webhook-response-fail-with-empty-speech-response – Richard Kennard May 15 '19 at 23:08
What I gave you above is the full response that is passed back. If your app passes the payload response back, then you should hear the Welcome message. So if you are using your own server, and any language like PHP or some other scripting language, then that "payload" response will work. If you want to display some different text from what is actually said, then you can use "displayText": "some text" right underneth "textToSpeech". Oh and don't forget to the add the comma to the end of "textToSpeech" if you do use "displayText". – sgodsell May 16 '19 at 23:30

Correct fulfillmentMessages format for DialogFlow Google Assistant

1 Answers1