Dialogflow textToSpeech fulfilment not reading aloud the text

Question

I am providing users with a response on an audio only device (e.g. google home), when I respond with a textToSpeech field within a simpleResponse, the speech is not read out in the simulator.

Has anyone experienced this and know how to fix?

I've tried different response types but none of them read out the textToSpeech field. Also tried ticking/unticking end conversation toggle in Dialogflow and expectUserInput true/false when responding with JSON to no avail.

The response is currently fulfilled by a webhook which responds with JSON v2 fulfilment blob and the simulator receives the response with no errors but does not read it out.

RESPONSE -

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
            "simpleResponse": {
              "textToSpeech": "Here are the 3 closest restaurants that match your criteria,"
            }
          }
        ]
      }
    }
  }
}

REQUEST -

{
  "responseId": "404f3b65-73a5-47db-9c17-0fc8b31560a5",
  "queryResult": {
    "queryText": "actions_intent_NEW_SURFACE",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "outputContexts": [
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/findrestaurantswithcuisineandlocation-followup",
        "lifespanCount": 98,
        "parameters": {
          "location.original": "Shoreditch",
          "cuisine.original": "international",
          "cuisine": "International",
          "location": {
            "subadmin-area": "Shoreditch",
            "subadmin-area.original": "Shoreditch",
            "subadmin-area.object": {}
          }
        }
      },
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/actions_capability_account_linking"
      },
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/actions_capability_audio_output"
      },
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/google_assistant_input_type_voice"
      },
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/actions_capability_media_response_audio"
      },
      {
        "name": "projects/my-project/agent/sessions/sessionId/contexts/actions_intent_new_surface",
        "parameters": {
          "text": "no",
          "NEW_SURFACE": {
            "@type": "type.googleapis.com/google.actions.v2.NewSurfaceValue",
            "status": "CANCELLED"
          }
        }
      }
    ],
    "intent": {
      "name": "projects/my-project/agent/intents/0baefc9d-689c-4c33-b2b8-4e130f626de1",
      "displayName": "Send restaurants to mobile"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en-us"
  },
  "originalDetectIntentRequest": {
    "source": "google",
    "version": "2",
    "payload": {
      "isInSandbox": true,
      "surface": {
        "capabilities": [
          {
            "name": "actions.capability.AUDIO_OUTPUT"
          },
          {
            "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
          },
          {
            "name": "actions.capability.ACCOUNT_LINKING"
          }
        ]
      },
      "requestType": "SIMULATOR",
      "inputs": [
        {
          "rawInputs": [
            {
              "query": "no",
              "inputType": "VOICE"
            }
          ],
          "arguments": [
            {
              "extension": {
                "@type": "type.googleapis.com/google.actions.v2.NewSurfaceValue",
                "status": "CANCELLED"
              },
              "name": "NEW_SURFACE"
            },
            {
              "rawText": "no",
              "textValue": "no",
              "name": "text"
            }
          ],
          "intent": "actions.intent.NEW_SURFACE"
        }
      ],
      "user": {
        "userStorage": "{\"data\":{}}",
        "lastSeen": "2019-04-12T14:31:23Z",
        "locale": "en-US",
        "userId": "userID"
      },
      "conversation": {
        "conversationId": "sessionId",
        "type": "ACTIVE",
        "conversationToken": "[\"defaultwelcomeintent-followup\",\"findrestaurantswithcuisineandlocation-followup\",\"findrestaurantswithcuisineandlocation-followup-2\"]"
      },
      "availableSurfaces": [
        {
          "capabilities": [
            {
              "name": "actions.capability.AUDIO_OUTPUT"
            },
            {
              "name": "actions.capability.SCREEN_OUTPUT"
            },
            {
              "name": "actions.capability.WEB_BROWSER"
            }
          ]
        }
      ]
    }
  },
  "session": "projects/my-project/agent/sessions/sessionId"
}

I expect the simulator to read out the result of textToSpeech but currently does not.

Completely off topic, but as a Python developer the last 7 lines are just painful to look at. — Malekai, Apr 12 '19 at 13:26
I don't think that just applies to Python ha, it's ugly in most languages. The whole JSON response fulfilment wasn't the way to go but hey ho — Ben Vaughan-Jones, Apr 12 '19 at 13:28
Can you update your question to include screen shots of the simulator when this is happening and the JSON from the "request", "response", and "debug" tabs? — Prisoner, Apr 12 '19 at 14:03
Which simulator are you using? The simulator inside of Dialogflow doesn't produce TTS. — Nick Felker, Apr 12 '19 at 14:21
@NickFelker I am using the simulator inside Dialogflow, I have managed to have it responding in voice for all of my other responses just not this particular one — Ben Vaughan-Jones, Apr 12 '19 at 14:26
@Prisoner not able to get the debug page on here but should be able to tell from the response and request JSON objects right? Is there anything in particular you are looking for? — Ben Vaughan-Jones, Apr 12 '19 at 14:57

Dialogflow textToSpeech fulfilment not reading aloud the text

0 Answers0