3

I developed a actions on google app which sends a rich response. Everything works fine in the Actions on Google simulator. Now I want to test it on my Google Home Mini but my rich responses are not told by the mini. I would like to ask if it is possible to send my rich response to the google home app? The home mini says something like "Ok, I found these hotels, look at the home app" and there are the rich responses?

lmaonuts
  • 125
  • 9

2 Answers2

4

You can't send users to the Home app, but you can direct them to the Assistant available through their phone. The process is roughly:

  1. At some point in the conversation (decide what is best for you, but when you have results that require display is usually good, or if the user says something like "Show me" or "Send this to my phone"), determine if they are on a device with a screen or not. You do this by using the app.getSurfaceCapabilities() method or by looking at the JSON in the originalRequest.data.surface.capabilities property. If they're using a screen, you're all set. But if not...

  2. Make sure they have a screen they can use. You'll do this by checking out the results from app.getAvailableSurfaces() or looking at the JSON in the (not fully documented) originalRequest.data.availableSurfaces array. If they don't have a screen, you'll need to figure out your best course of action. But if they do have a screen surface (such as their phone, currently) available...

  3. You can request to transfer them to the new surface using the app.askForNewSurface() method, passing a message explaining why you want to do the switch, a message that will appear as a notification on the device, and what surface you need (the screen).

  4. If the user approves, they'll get the notification on their mobile device (using that device's normal notification system). When they select the notification, the Assistant will open up and will send your Action an Event called actions_intent_NEW_SURFACE. You'll need to create an Intent that handles this Event and forwards it to your webhook.

  5. Your webhook should confirm that it is on a useful surface, and then proceed with the conversation and send the results.

You can see more about handling different surfaces at https://developers.google.com/actions/assistant/surface-capabilities

Prisoner
  • 49,922
  • 7
  • 53
  • 105
  • Thanks for clarification. Do you know if it is possible to send data, for example pictures, to the phone if you say "Ok Google, send this to my phone". The Assistant app may then display the pictures... – lmaonuts Dec 18 '17 at 15:39
  • It depends where they say that and why. I've updated my answer to point out that if they say something like that while talking to your action, that might be a great time to do the handoff. – Prisoner Dec 18 '17 at 16:01
1

Rich responses can appear on screen-only or audio and screen experiences. They can contain the following components:

  • One or two simple responses (chat bubbles)
  • An optional basic card
  • Optional suggestion chips
  • An optional link-out chip
  • An option interface (list or carousel)

So you need to make sure that the text response is containing all the details for cases like voice only (e.g. Google home/mini/max).

However, if your users are using the assistant from a device with a screen, you can offer them a better experience with the rich responses (e.g. suggestion chips, links etc').

Ido Green
  • 2,795
  • 1
  • 17
  • 26