Google Home dialogFlow V2 API mediaResponse not working

Question

I decided to upgrade my Google Assistant action to use "dialogFlow V2 API" and my webhook returns an object like this

{
    "fulfillmentText": "Testing",
    "fulfillmentMessages": [
        {
            "text": {
                "text": [
                    "fulfillmentMessages text attribute"
                ]
            }
        }
    ],
    "payload": {
        "google": {
            "richResponse": {
                "items": [
                    {
                        "mediaResponse": {
                            "mediaType": "AUDIO",
                            "mediaObjects": [
                                {
                                    "name": "mediaResponse name",
                                    "description": "mediaResponse description",
                                    "largeImage": {
                                        "url": "https://.../640x480.jpg"
                                    },
                                    "contentUrl": "https://.../20183832714.mp3"
                                }
                            ]
                        },
                        "simpleResponse": {
                            "textToSpeech": "simpleResponse: testing",
                            "ssml": "simpleResponse: ssml",
                            "displayText": "simpleResponse displayText"
                        }
                    }
                ]
            }
        }
    },
    "source": "webhook-play-sample"
}

But I get an error message saying my action it is not available, is mediaResponse supported by V2?, should I format my object differently?, also, when I remove "mediaResponse" object works just fine and the assistant will speak the simpleResponse part.

This action was re-created this Mid March 2018 and I read about May deadline and that is why I decide to upgrade to V2, do you think I should go back to V1, I know I will have to delete it and re-created but that is fine. This is a link to the JSON object I see in the debug tab. Thanks once again

I set "API V2" in my action dialogFlow console, this is a screenshot of that setting

Here is an screenshoot of my action's integration -> Google Assistant

Thanks Allen, Yes I do have "expectUserResponse": false, I added the suggestion object you recommended but, unfortunately nothing changed, I am still getting this error Simulator debug tag details

Which "V2" are you talking about" Actions on Google V2 or Dialogflow V2? (Where did you change the setting to use V2?) — Prisoner, Mar 22 '18 at 17:58
Ok, can you go to the "Integrations" page and select "Google Assistant" and include a screen shot of what appears there? — Prisoner, Mar 22 '18 at 19:48

Prisoner · Answer 1 · 2018-03-25T13:04:15.653

First of all - this is not a problem with Dialogflow V2. You also seem to be confusing the sunset of Actions on Google V1 with the release of Dialogflow V2 - they are two different creatures completely. If your project was using AoG V1, there would be a setting on the Actions integration screen, and thee isn't.

It is fine if you want to move to Dialogflow V2, but it isn't required. Media definitely works under Dialogflow V2.

The array of items must include a simpleResponse item first, before any of the other items in the RichResponse. (You also shouldn't include both ssml and textToSpeech - just one of them.) You also don't need the fulfillmentText and fulfillmentMessages components, since those are provided by the richResponse.

You also need to include suggestions chips unless you have set expectUserResponse to false. Somewhere in the simulator debug is probably a block that says

      {
        "name": "MalformedResponse",
        "debugInfo": "expected_inputs[0].input_prompt.rich_initial_prompt: Suggestions must be provided if media_response is used..",
        "subDebugEntryList": []
      }

So something more like this should work:

{
    "payload": {
        "google": {
            "richResponse": {
                "items": [
                    {
                        "simpleResponse": {
                            "textToSpeech": "simpleResponse: testing",
                            "displayText": "simpleResponse displayText"
                        },
                        "mediaResponse": {
                            "mediaType": "AUDIO",
                            "mediaObjects": [
                                {
                                    "name": "mediaResponse name",
                                    "description": "mediaResponse description",
                                    "largeImage": {
                                        "url": "https://.../640x480.jpg"
                                    },
                                    "contentUrl": "https://.../20183832714.mp3"
                                }
                            ]
                        }
                    }
                ]
                "suggestions": [
                    {
                        "title": "This"
                    },
                    {
                        "title": "That"
                    }
                ]
            }
        }
    },
    "source": "webhook-play-sample"
}

Thanks Allen, but that did not fix the issue, I am still getting "{myActionName} isn't responding right now, try again later" ;( — igorvargas, Mar 22 '18 at 17:00
Are you testing this with the simulator? Can you update the quesiton to include screen shots of the simulator showing the error and showing the Debug tab? — Prisoner, Mar 22 '18 at 17:20
Don't read too much into that since the internal representation isn't necessarily the same as the API names we see. How old is the Dialogflow project you're working with? Did you create it before last May? (And showing the entire screen of the simulator will help - just that fragment doesn't help a lot.) — Prisoner, Mar 22 '18 at 17:42
Updating the original question is better than trying to provide links and updates in the comments. — Prisoner, Mar 22 '18 at 17:58
Updated answer. (Two parts, one about V1 vs V2 and one about some things missing from the JSON.) — Prisoner, Mar 25 '18 at 13:04

Google Home dialogFlow V2 API mediaResponse not working

1 Answers1