0

I've been trying to implement speech recognition on android-driven smart-glasses Vuzix M300 (there's a built-in solution already, but it doesn't support the language I need). I'm trying to run just a pocketsphinx-android demo-app (which in that very configuration works fine on my smartphone. Btw, I've disabled the wear module). The problem is that onPartialResult() method is never called, no audiostream is processed. The error I instantly get is:

E/sound_trigger_hw: int sound_trigger_open_for_streaming(): DSP is not currently streaming E/AudioStreamIn: intel_audio::StreamIn::StreamIn(intel_audio::Device*, audio_io_handle_t, uint32_t, audio_source_t, audio_devices_t): Failed to open DSP for streaming

After async task which sets up rcognizer is finished app crushes with error:

09-26 15:26:27.779 9059-9059/edu.cmu.sphinx.pocketsphinx A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0xc in tid 9059 (nx.pocketsphinx)

09-26 15:26:27.881 293-293/? A/DEBUG: * *

09-26 15:26:27.882 293-293/? A/DEBUG: Build fingerprint: 'intel/m300/vm300:6.0.1/MMB29M/201709151537:user/1.2.2'

09-26 15:26:27.882 293-293/? A/DEBUG: Revision: '0'

09-26 15:26:27.882 293-293/? A/DEBUG: ABI: 'x86'

09-26 15:26:27.882 293-293/? A/DEBUG: pid: 9059, tid: 9059, name: nx.pocketsphinx >>> edu.cmu.sphinx.pocketsphinx <<<

09-26 15:26:27.882 293-293/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc

09-26 15:26:27.914 293-293/? A/DEBUG: eax f7ba6650 ebx f3d15964 ecx 00000000 edx ffd1bc78

09-26 15:26:27.914 293-293/? A/DEBUG: esi ffd1bc78 edi f7ba66dc

09-26 15:26:27.914 293-293/? A/DEBUG: xcs 00000023 xds 0000002b xes 0000002b xfs 00000007 xss 0000002b

09-26 15:26:27.915 293-293/? A/DEBUG: eip f3aed3bf ebp ffd1bcb8 esp ffd1bc20 flags 00210292

09-26 15:26:27.920 293-293/? A/DEBUG: backtrace:

09-26 15:26:27.921 293-293/? A/DEBUG: #00 pc 005603bf /system/lib/libart.so (_ZN3art6Thread28PopDeoptimizationReturnValueEv+31)

09-26 15:26:27.921 293-293/? A/DEBUG: #01 pc 0014536b /system/lib/libart.so (_ZN3art9ArtMethod6InvokeEPNS_6ThreadEPjjPNS_6JValueEPKc+667)

09-26 15:26:27.921 293-293/? A/DEBUG: #02 pc 0052feb8 /system/lib/libart.so (_ZN3art12InvokeMethodERKNS_33ScopedObjectAccessAlreadyRunnableEP8_jobjectS4_S4_j+1752)

09-26 15:26:27.921 293-293/? A/DEBUG: #03 pc 004a7e41 /system/lib/libart.so (_ZN3artL13Method_invokeEP7_JNIEnvP8_jobjectS3_S3_+81)

09-26 15:26:27.921 293-293/? A/DEBUG: #04 pc 72a83ca4 /data/dalvik-cache/x86/system@framework@boot.oat (offset 0x1f10000)

09-26 15:26:28.052 293-293/? A/DEBUG: Tombstone written to: /data/tombstones/tombstone_03

09-26 15:26:28.052 293-293/? E/DEBUG: AM write failed: Broken pipe

09-26 15:26:28.379 205-205/? E/lowmemorykiller: Error opening /proc/9059/oom_score_adj; errno=2

And when I get into public boolean startListening(String searchName){} method in class SpeechRecognizer.class I start getting this error ifinitely:

E/AudioStreamIn: setDevice: invalid input device 4
E/AudioIntelHal: read: No sound trigger handle found for input source 6

E/AudioFlinger: read failed: framesRead=-22

as I hit this code: this.recognizerThread.start();

The point is that standard android AudioRecord works fine on glasses, no problems with micro occures, but pocketsphinx does not simply see or process the audiostream. All permissions are granted, the decoder is set and ready, but no success.

Min SDK version: API 23: Android 6.0 (Marshmallow)

Compile SDK version: Vuzix M300 SDK (API 23)

Build Tools Version: 26.0.1

(Also tried min API 19, Compile SDK API 25, Build tool 25.0.0 successfully on smatphone, with no success on glasses)

Any help is highly appreciated!

UPD: well, if audioRecord.read(buffer, 0, buffer.length); is called after audioRecord.startRecording(); everything freezes when I hit 

native_read_in_short_array(audioData, offsetInShorts, sizeInShorts,
                readMode == READ_BLOCKING);

in AudioRecord class. There I have no clues..

Sophour
  • 106
  • 9
  • Possible duplicate of [Android Studio Emulator E/AudioFlinger error message](https://stackoverflow.com/questions/36210348/android-studio-emulator-e-audioflinger-error-message) – Nikolay Shmyrev Sep 26 '17 at 22:35
  • @NikolayShmyrev that didn't work for me. I have microG services preinstalled instead of google services, the built-in voice recognition is disabled and absent in running services, default assistant app isn't set and there's no any at all... – Sophour Sep 27 '17 at 08:02
  • Something is still listening and holding access to the audio device. – Nikolay Shmyrev Sep 27 '17 at 13:45
  • @NikolayShmyrev, if so I wouldn't have accessed the micro and made an audio record with another app. Why would android.media.MediaRecorder fnd android.media.AudioRecord worked, when pocketsphinx did not? May it be an issue with my micro record parameters? Like, with stereo/mono channel or sample rate? How to tune it with respect to pocketsphinx lib then? – Sophour Sep 27 '17 at 14:44
  • Pocketsphinx requires 16khz mono. You can check if AudioRecord works with those params easily. – Nikolay Shmyrev Sep 27 '17 at 19:58
  • @NikolayShmyrev, well, I've checked. Different configs like `AudioRecord(MediaRecorder.AudioSource.MIC, 16000, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, 1280);` and `AudioRecord(MediaRecorder.AudioSource.DEFAULT, 16000, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_DEFAULT, 1280);` has been initialized. Could it be some specific encoding in pocketsphinx? Cause once while `AudioRecord.getMinBufferSize()` I've got a negative buffer size provided 16kHz and ENCODING_PCM_16BIT – Sophour Sep 28 '17 at 08:47
  • I made a mistake, I got a negative buffer size due to ENCODING_DEFAULT, if it matters (it doesn't)* Although I've found which encoding does pocketsphinx-android uses and it's ok, but still I don't get an audiostream from micro for SpeechRecognizer to read. I'm running out of clues... – Sophour Sep 28 '17 at 13:55
  • Pocketsphinx uses AudioSource.VOICE_RECOGNITION. – Nikolay Shmyrev Oct 14 '17 at 14:04

1 Answers1

1

As Nicolay said, the problem appeared due to audio source. Our devise's built-in voice recognition sys has been blocking VOICE_RECOGNITION source somehow even being switched off. After replacing VOICE_RECOGNITION by MIC source in AudioRecord initialization within Pocketsphinxe's android sources everything worked.

Sophour
  • 106
  • 9