SpeechRecognizer broken after Google Glass XE17 update -- how to work around?

Question

Standard Android SpeechRecognizer was working perfectly on Google Glass XE16 - XE16.2
Then the XE17 update suddenly broke everything, with the following error and no callbacks to the Listener anymore:

E/AndroidRuntime(6321): FATAL EXCEPTION: main 
E/AndroidRuntime(6321): Process: com.google.glass.voice, PID: 6321 
E/AndroidRuntime(6321): java.lang.NullPointerException: VoiceEngine.startListening: voiceConfig cannot be null
E/AndroidRuntime(6321): at com.google.glass.predicates.Assert.assertNotNull(Assert.java:68)
E/AndroidRuntime(6321): at com.google.glass.voice.VoiceEngine.startListening(VoiceEngine.java:650)
E/AndroidRuntime(6321): at com.google.glass.voice.VoiceService$VoiceServiceBinder.startListening(VoiceService.java:116)
E/AndroidRuntime(6321): at com.google.glass.voice.GlassRecognitionService.attachCallback(GlassRecognitionService.java:272)
E/AndroidRuntime(6321): at com.google.glass.voice.GlassRecognitionService.onStartListening(GlassRecognitionService.java:216)
E/AndroidRuntime(6321): at android.speech.RecognitionService.dispatchStartListening(RecognitionService.java:98)
E/AndroidRuntime(6321): at android.speech.RecognitionService.access$000(RecognitionService.java:36)
E/AndroidRuntime(6321): at android.speech.RecognitionService$1.handleMessage(RecognitionService.java:79)
E/AndroidRuntime(6321): at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(6321): at android.os.Looper.loop(Looper.java:149)

By disassembling the GlassVoice.apk (located in /system/priv-app/), I was able to find out that you can add these two extras to your SpeechRecognizer's Intent, and it fixes the NullPointerException :

 
//mSpeechIntent.putExtra( GlassSpeechRecognizer.EXTRA_VOICE_CONFIG_NAME, "Toto");
mSpeechIntent.putExtra( "voiceConfigName", "Toto");
//mSpeechIntent.putExtra( GlassSpeechRecognizer.EXTRA_VOICE_COMMANDS, new String[]{"red","green","blue"} );
mSpeechIntent.putExtra( "extraVoiceCommands", new String[] {"red","green","blue"}); // Command phrases allowed!

The problem is, as they moved the GlassVoice.apk in a private space, that you cannot load any class of it from your own app. I did not find out how to work around this -- if you know how, help would be much appreciated!

With that, I can see in the logs that one phrase has been recognized when I speak, but I don't have any callback on the usual listener.

Some of the interesting logs (word "red" recognized, but no callbacks):

I/RecognizerController(1818): attachVoiceInputCallback
W/RecognizerController(1818): queueingGrecoListener was null in attachVoiceInputCallback
...
I/VoiceEngine[20daf4b4](1818): Hotword recognizer triggered a recognition result
...
I/SavedAudioStorage(1818): Saved SavedAudioRecord [id=null, filename=/data/data/com.google.glass.voice/recorded_audio/20140508_171311_197.pcm, recognized=true, synced=false, timestamp=1399594400939, recognizedCommands=red:2000:2620, sampleRate=16000]

How can we work around this?

EDIT WITH ANSWER!

Thanks to @pscholl, I found a workaround. If your app is simple and does not include many jars, just look at @pscholl solution.

In my case, adding GlassVoice-xe17.apk, which contains thousands of methods, made my app hit the infamous "64K methods" Android ceiling. If you don't know what this terrible limit is, just think about the "640K in MS-DOS", reinvented by Android (Dalvik VM).

First I tried to turn ProGuard on, in order to shrink my app, by removing all unused methods: the problem is that it is a nightmare to configure, and after hours of failed attempts, I was still having unexpected classes missing and cryptic errors.

So I turned to Dex loading, as explained here: https://github.com/mmin18/Dex65536. The problem is that GlassVoice.apk includes all com.google.common package, that I also use through Guava. In the Dex65536 solution, the classloader loads the class from the external APK first (nice hack, that you can't turn around), and therefore it was not working.

I ended up coding a "class pre-loader", invoked before the additional Dex ClassLoader is added. Drop me a line if you're interested in the code (but it's far from being optimal).

That was tough and ugly! I hope the Glass Dev Team will solve the voice problem soon, with a neat solution -- not like this one :-)

score 1 · Accepted Answer · answered May 09 '14 at 11:06

1

Still works fine for me on XE17 with the method described here: Glass voice command nearest match from given list, maybe you called the initial setVoiceConfig with a null

answered May 09 '14 at 11:06

pscholl

522
4
6

Hi @pscholl, two remarks: 1. This thread opens an Activity, whereas we need to have the SpeechRecognizer work without leaving the current card 2. I applied exactly what was said and got errors "Class not found" on VoiceConfig when running the code on XE17. It looks like when the apk is in /system/priv-app, you cannot load the classes from your app. Or am I missing something? – Eric May 09 '14 at 20:03
This is the exact error I get when I simply try in my code: `VoiceConfig mVoiceConfig = new VoiceConfig("MyVoiceConfig", new String[] {"red","green","blue"} );` `05-09 14:21:29.372: E/AndroidRuntime(31742): java.lang.NoClassDefFoundError: com.google.glass.voice.VoiceConfig` @pscholl have you rooted your Glass and done any modification to the off-the-shelf XE17? Thanks for your help! – Eric May 09 '14 at 21:23
I'm curious why this does not work for you. It also works on a non-rooted for me, without any modifications. Are you packing the decompiled GlassVoice.apk into the resulting .apk of the application? This example https://github.com/pscholl/glass_snippets/tree/master/esslib contains a full-fledged demo, find the implementation of a VoiceMenu in the lib/ folder and the demo in voiceMenu/. – pscholl May 10 '14 at 19:07
@Eric pscholl's method is working for me in the interim. i still prefer your implementation, but his method works for now. – Cole May 12 '14 at 14:52
Thank you @pscholl, great examples and clear code that helped me a lot! In my case it was not so easy actually, I'll edit my question to explain. That was a tough struggle :( – Eric May 14 '14 at 10:51
@eric ha, never heard of that 64k limit (crazy). Wouldn't removing some of the unnecessary classes in GlassVoice-xe17.jar work in your case? After all it's just a zipfile which you can repack which less content – pscholl May 14 '14 at 17:24
@pscholl yeah I could have done that, but it would have been a tedious process to keep the right dependencies. And so if they break the recognition again, now I have a generic way to solve it ;-) – Eric May 16 '14 at 18:35
@pscholl Hi Phil, I saw the following method call in your code: `mVoiceInputHelper.setVoiceConfig( mVoiceConfig, false );` Do you have any idea what this boolean means? – Eric May 16 '14 at 18:36
IIRC it states wether audio should be saved – pscholl May 17 '14 at 11:17

score 0 · Answer 2 · answered May 13 '14 at 13:44

Alternatively if you can just fallback to the android speech recognizer. This is not nearly as elegant as the Glass code.

private void displaySpeechRecognizer() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "What is your favorite color (e.g. Red, Blue Green)");
        startActivityForResult(intent, SPEECH_REQUEST);
}
private   @Override
    protected void onActivityResult(int requestCode, int resultCode,
                                    Intent data) {
        if (requestCode == SPEECH_REQUEST && resultCode == RESULT_OK) {
            String results = StringUtils.join(data.getStringArrayListExtra(
                    RecognizerIntent.EXTRA_RESULTS));
             //TODO: your switch based on string results.  Not you may need to do a fuzzy match based on confidence scores.
         }   
}

http://developer.android.com/reference/android/speech/RecognizerIntent.html

Hi John, thank you for your answer. This code opens a new activity, so you have to leave the card you're on. In my case I strongly need to stay on the same screen. — Eric, May 14 '14 at 10:53

SpeechRecognizer broken after Google Glass XE17 update -- how to work around?

2 Answers2

Linked