How to implement Media Response with Dialogflow

Question

I am using Dialogflow and would like to implement a media response in my project. I have tried to do so using the following link: https://developers.google.com/actions/assistant/responses but have been unsuccessful. How to do the implementation?

const functions = require('firebase-functions')
const {dialogflow} = require('actions-on-google')

const TEST = 'test'

const app = dialogflow({
    debug: true,
})

app.intent('test', (conv) =>{
    conv.ask('we will now play a song for you');

    conv.ask(new MediaObject({
        name: 'Jazz in Paris',
        url: 'https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3',
    }));
    conv.ask(new MediaResponse({
        mediaObject: 'Jazz in Paris',
        mediaType: AUDIO,
    }));
});


exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app)

When you say you have been "unsuccessful", what have you tried? Have you written any code? If so, can you update your question with the code that you tried? — Prisoner, Jan 03 '19 at 23:42
Please don't use a picture, since it is really difficult to read, to reproduce any problems you may have, and to use it as a basis for examples. Please copy and paste the text into the StackOverflow question and format it (see https://stackoverflow.com/editing-help#code) — Prisoner, Jan 04 '19 at 00:11
And you say you don't know how to "initiate" the response. What do you mean? — Prisoner, Jan 04 '19 at 00:13
I have made a media object and a media response (I do not know if these are initialized correctly). But now I want my intent to play the media object that is in the media response, and am unsure of how to do this. — user10687878, Jan 04 '19 at 00:27

Prisoner · Answer 1 · 2019-01-04T01:15:00.903

2

You just need to add the MediaObject. You don't need to add the MediaResponse object since the library will add it for you.

You will, however, need to make sure you load the MediaObject object as part of your call to require(). You'll also need to load the Suggestions object since you need to provide suggestions to advance the conversation if the user chooses to interrupt your audio.

So your code can look something like this:

const functions = require('firebase-functions')
const {
  dialogflow,
  MediaObject,
  Suggestions
} = require('actions-on-google')

const app = dialogflow({
    debug: true,
})

app.intent('test', (conv) =>{
    conv.ask('we will now play a song for you');

    conv.ask(new MediaObject({
        name: 'Jazz in Paris',
        url: 'https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3',
    }));

    conv.ask(new Suggestions(['suggestion 1', 'suggestion 2']));

});


exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app)

edited Jan 04 '19 at 01:15

answered Jan 04 '19 at 00:53

Prisoner

49,922
7
53
105

Thanks! I've made the suggestion changes, but now I get the following error when I call the intent: "MalformedResponse expected_inputs[0].input_prompt.rich_initial_prompt: Suggestions must be provided if media_response is used.." – user10687878 Jan 04 '19 at 01:09
1

Answer is updated. As the error message says, you need to include suggestion chips that will advance the conversation if the user wishes to interrupt the audio. – Prisoner Jan 04 '19 at 01:15
Awesome! If an answer has helped, accepting and/or upvoting it is always appreciated. – Prisoner Jan 04 '19 at 03:00
I've recently tried this feature out on my actual Google Home device, and for some reason when I say the hot word, the device does not pick it up. However, when I WROTE out the response in the simulator, it works. Any suggestions for why this is happening? The code I have implemented is the same as above. – user10687878 Jan 11 '19 at 18:42
Not offhand, but it doesn't sound related to the code specifically. I'd ask a new StackOverflow questions that provide as many specific details as you can. – Prisoner Jan 11 '19 at 19:00
It seems like I need to say 'OK Google' first, and then the hotword for it to be activated. Any idea how to disable the 'OK GOOGLE' requirement, so that Google just picks up the hot word? – user10687878 Jan 13 '19 at 22:54
"OK Google" or "Hey Google" *is* the hotword. For voice-driven devices, that is what opens the microphone once it closes it. Media playback automatically closes the microphone. – Prisoner Jan 13 '19 at 23:46
is there a way to re-open the microphone, while a media playback is occurring, with a hotword that is different than 'ok google' or 'hey google' ? – user10687878 Jan 13 '19 at 23:49
No, it is done this way for security and consistency. Users can use "Hey Google" at any point to interrupt any response. – Prisoner Jan 14 '19 at 00:05
If you think you need something else, I suggest you open a new question and explain what you're trying to do in detail, what isn't working, and why. Perhaps we can come up with another solution. If this answer has helped, accepting and/or upvoting it is always appreciated. – Prisoner Jan 14 '19 at 01:07

How to implement Media Response with Dialogflow

1 Answers1