1

I'm having an issue with my Google Assistant Action and using it in the Google Assistant Mobile app.

I am trying to play a tracklist of 1-3 minute mp3s using Media Responses and callbacks and it is working perfectly in the simulator and on my Google Home Mini, but not on the Google Assistant app on my phone.

What I've noticed happening is the MediaResponse callback isn't sent when I test on iPhone. The first MediaResponse will play but then the app is silent. It doesn't exit my action, though, it leaves the mic open and when I try to talk to it again whatever I say is sent to my action. This part is very similar to Starfish Mint's problem, though mine seems to work on my Google Home device. They said they fixed it by

"After waiting 6 months, We manage to solve it ourselves. On MEDIA_FINISHED, we must return Audio text within your media response to get subsequent MEDIA_FINISHED event. We tested this with playlist of 100 media files and it plays like a charm."

though I'm not entirely sure what that means.

This might be an obvious answer to my question but where it says: Media responses are supported on Android phones and on Google Home , does this mean that they aren't supported on iPhone and that's the issue? Are there any workarounds for this, like using a Podcast action or something?

I have tried another audio playing app, the Music Player Sample app which is one of Google's sample Dialogflow apps and it also doesn't work on my phone though does in the other places. Maybe it is just an iPhone thing?

The thing that I find confusing, though, is when I look at the capabilities of the action on my phone: conv.surface.capabilities.has("actions.capability.MEDIA_RESPONSE_AUDIO")it includes actions.capability.MEDIA_RESPONSE_AUDIO in its capabilities. If it didn't have this I would be more inclined to believe it doesn't include iPhones but it seems weird that it would have it in the capabilities but then not work.

Here's the code where I am playing the first track:

app.intent('TreatmentResponse', (conv, context, params) => {
    var treatmentTracks = [{url: 'url', name: 'name'},{url: 'url', name: 'name'}];
    var result = playNext(treatmentTracks[0].url, treatmentTracks[0].name);
    var response = result[0];
    conv.data.currentTreatment = 'treatment';
    conv.data.currentTreatmentName = 'treatmentName';
    conv.data.treatmentPos = 1;
    conv.data.treatmentTracks = treatmentTracks;
    conv.ask("Excellent, I'll play some tracks in that category.");
    conv.ask(response);
    conv.ask(new Suggestions(['skip']));
});

and here is my callback function:

app.intent('Media Status', (conv) => {
  const mediaStatus = conv.arguments.get('MEDIA_STATUS');
  var { treatmentPos, treatmentTracks, currentTreatment, currentTreatmentName } = conv.data;
  if (mediaStatus && mediaStatus.status === 'FINISHED' && treatmentPos < treatmentTracks.length) {
    playNextTrack(conv, treatmentPos, treatmentTracks);
  } else {
    endConversation(conv, currentTreatment);
  }
});

Here's playNextTrack()

function playNextTrack(conv, pos, medias) {
  conv.data.treatmentPos = pos+1;
  var result = playNext(medias[pos].url, medias[pos].name);
  var response = result[0];
  var ssml = result[1];
  conv.ask(ssml);
  conv.ask(response);
  conv.ask(new Suggestions(['skip']));
}

and playNext()

function playNext(url, name) {
  const response = new MediaObject({
    name: name,
    url: url,
  });
  var ssml = new SimpleResponse({
      text: 'Up next:',
      speech: '<speak><break time="1" /></speak>'
    });
  return [response, ssml];
}

The other issue is when the MediaResponse is playing on my iPhone if I interrupt it to say "Next" or "Skip", rather than using my "NextOrSkip" intent like it does in the simulator and on the Google Home Mini, it just says "sure" or "alright" [I don't have that in my code anywhere] and then is silent (and listening).

Juliet
  • 11
  • 2

0 Answers0