Can't start pocketsphinx in Unity3d game

Question

I am trying to integrate PocketSphinx into my Unity3d project.
In order to do this, I have integrated the Android demo project into the Android Studio export of the Unity3d project.

When trying to the app, I get an exception saying that the following function retured -1.

public static final native void Decoder_setSearch(long var0, Decoder var2, String var3);

This method is being called when I start listening for voice commands:

recognizer.startListening("hello app", 10000);

Why does this happen? How can I resolve it? It works perfectly in the PocketSphinx example project.

How I've set up the Recognizer:

Assets assets = new Assets(MyActivity.this);
File assetDir = assets.syncAssets();

recognizer = defaultSetup()
                .setAcousticModel(new File(assetDir, "en-us-ptm"))
                .setDictionary(new File(assetDir, "cmudict-en-us.dict"))
                .setRawLogDir(assetDir)
                .setKeywordThreshold(1e-45f)
                .setBoolean("-allphone_ci", true)
                .getRecognizer();

recognizer.addListener(MyActivity.this);
recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);

...and more info from logcat (complete file):

01-29 18:01:12.989 4237-4237/com.myName.pocketSphinxTest I/SpeechRecognizer: Start recognition "hello app"
01-29 18:01:12.989 4237-4237/com.myName.pocketSphinxTest D/AndroidRuntime: Shutting down VM
01-29 18:01:12.989 4237-4237/com.myName.pocketSphinxTest W/dalvikvm: threadid=1: thread exiting with uncaught exception (group=0x4157fd58)
01-29 18:01:12.989 4237-4237/com.myName.pocketSphinxTest E/AndroidRuntime: FATAL EXCEPTION: main
                                                                             Process: com.myName.pocketSphinxTest, PID: 4237
                                                                             java.lang.Error: FATAL EXCEPTION [main]
                                                                             Unity version     : 5.2.2f1
                                                                             Device model      : Osterhout_Design_Group R7-W
                                                                             Device fingerprint: R7/apq8084/apq8084:4.4.4/KTU84P/ODG_R7_V3.1.12:user/release-keys
01-29 18:01:13.019 4237-4237/com.myName.pocketSphinxTest I/Process: Sending signal. PID: 4237 SIG: 9

You incorrectly specified the path to assets in setSearch it seems. You can find details in logcat output — Nikolay Shmyrev, Apr 21 '16 at 16:40
@NikolayShmyrev I'm pretty sure I didn't. Please check out the additional code I appended to my question. :) — 000000000000000000000, Apr 22 '16 at 06:05
@NikolayShmyrev ...and here's the complete logcat file: http://snippi.com/raw/e8tjh6k — 000000000000000000000, Apr 22 '16 at 10:24
And what is the value of KWS_SEARCH? Why do you start "hello app" search and not KWS_SEARCH? — Nikolay Shmyrev, Apr 22 '16 at 12:45
It actually has this value, I replaced "hello app" with "KWS_SEARCH". Obviously it doesn't make a difference. — 000000000000000000000, Apr 22 '16 at 12:57
@NikolayShmyrev Where can I even find the method Decoder_setSearch(long var0, Decoder var2, String var3), so I can try to understand why it returns -1? — 000000000000000000000, Apr 25 '16 at 07:53

score 1 · Answer 1 · answered Apr 25 '16 at 08:03

setSearch is called in startListening with the name of the search. The name of the search is "hello app" in your case. If the search "hello app" is not added before it returns -1 and throws exception. The source is available in pocketsphinx-android sources.

The error generally means you didn't add the search with the name "hello app" before, from your code you say that KWS_SEARCH has the same value of "hello app", but I think there is a typo. You need to double-check that you use the same string for the search name. I would recommend you to replace "hello app" string with KWS_SEARCH in startListening call.

Thanks, but I've found the error somewhere else. onPartialResult(Hypothesis) was called with a null parameter and I didn't check for that. — 000000000000000000000, Apr 25 '16 at 13:55

Can't start pocketsphinx in Unity3d game

1 Answers1