Getting google assistant to respond in another language

Question

I've set up an app/agent in google's Dialogflow. I've gotten my assistant to talk to it, and perform Web-hooks to a service that i've set up.

The user talks to the google assistant in English, the service Google is talking to, essentially just randomizes a swedish word, and presents it back as text to speech in swedish.

For instance, "Ok Google, talk to testapp today's word." and google responds with "Dagens ord: köttfärs".

However, the google voice speaking back to the user, is stuck in English and I can't seam to change it to Swedish response. I want the app to listen in English, but respond in Swedish if that makes sense, it's a training exercise for english friends to learn swedish words.

I found some API documentation mentioning locale, so I tried fiddling with it but that seams to be bound to the client/agent's source language determaining the outcome voice..

Here's the response data (in python format):

response_headers = {}
response_headers[b'Content-Type'] = b'application/json'
response_headers[b'Google-Assistant-API-Version'] = b'v2'

response_data = {
    "fulfillmentText": TTS,
    "locale" : "sv-SE",
    "fulfillmentMessages": [
        {
            "card": {
                "title": TTS,
                "subtitle": TTS,
                "imageUri": "https://assistant.google.com/static/images/molecule/Molecule-Formation-stop.png",
                "buttons": [
                    {
                        "text": "Turn it off",
                        "postback": "https://service.com/google_home.py?state=off"
                    }
                ]
            }
        }
    ],
    "source": "service.com",
    "payload": {
        "google": {
            "expectUserResponse": False,
            "locale" : "sv-SE",
            "richResponse": {
                "items": [
                    {
                        "simpleResponse": {
                            "textToSpeech": TTS,
                            "locale" : "sv-SE"
                        }
                    }
                ]
            }
        }
    }
}

Is there a way to tell the assistant that it should switch language context on reply, not depending on input? Adding locale above seamed to do very little so I guess I'm way off here?

I should perhaps mention that TTS is the phrase "Dagens ord: köttfärs" for instance.

score 1 · Accepted Answer · answered Jan 04 '20 at 20:17

Unfortunately, there is no directly supported way to have the Assistant reply in a different language than the conversation is setup for (which is what it expects the input to be).

For simple things, instead of using text-to-speech, you can use SSML to play pre-recorded audio. Something like

   <speech>
     <audio src="https://example.com/audio/dagens-ord-kottfars.mp3">
       <desc>Dagens ord: köttfärs</desc>
     </audio>
   </speech>

If you needed this to be more dynamic, you create another webhook that takes the text that you want and uses something like Google's text-to-speech API to create an mp3 audio. You can then provide this URL as the src of the audio. (Unfortunately, Google doesn't seem to allow for data URLs either.)

(And yes, before you say it, I agree, I wish they'd support multi-lingual responses already.) — Prisoner, Jan 04 '20 at 20:18
Great workaround to be honest, introduces some lag obviously but that's acceptable. Yea I wish they supported this, since they have the ability to. But for now, good workaround and you have my thanks! — Torxed, Jan 04 '20 at 22:51

Getting google assistant to respond in another language

1 Answers1