Can I use OpenEars (or other) to listen to & analyze speech from the Apple Watch microphone?

Question

Is it possible to use OpenEars (or other package) to access speech from the Apple Watch's microphone?

I wish to build an app that is able to listen to speech, using the watch's microphone, and spots specific keywords.

Answer 1

No, it's not possible at the moment to access directly the Apple Watch microphone. I found the answer in the official developer forum.

WatchKit currently doesn't provide access to the watch's microphone. You do, however, have access to the iPhone's microphone from the WatchKit extension.

You can use the dictation system offered by Apple explained here: WatchKit: Speech to text conversion in WatchKit Apps

Answer 2

As of WatchOS 2.1, and iOS 9, I have been able to what you propose, in 2 different ways:

OPTION 1 - RECORD WAV FILE AND UPLOAD TO ASR SERVER I recorded and saved a WAV file to the apple watch. After that I uploaded the file to a paid Speech Recognition provider and everything worked fine! Here is the code to record, replace the UI updating lines of code (and the debug ones) with your own:

//RECORD AUDIO SAMPLE
    var saveUrl: NSURL? //this var is initialized in the awakeWithContext method//
    func recordAudio(){
        let duration = NSTimeInterval(5)
        let recordOptions =
        [WKAudioRecorderControllerOptionsMaximumDurationKey : duration]
       // print("Recording to: "+(saveUrl?.description)!)
        //CONSTRUCT AUDIO FILE URL
        let fileManager = NSFileManager.defaultManager()
        let container = fileManager.containerURLForSecurityApplicationGroupIdentifier("group.artivoice.applewatch");
        let fileName = "audio.wav"
        saveUrl = container?.URLByAppendingPathComponent(fileName)
        presentAudioRecorderControllerWithOutputURL(saveUrl!,
            preset: .WideBandSpeech,
            options: recordOptions,
            completion: { saved, error in
                if let err = error {
                    print(err.description)
                    self.sendMessageToPhone("Recording error: "+err.description)
                }
                if saved {
                    self.btnPlay.setEnabled(true)
                    self.sendMessageToPhone("Audio was saved successfully.")
                    print("Audio Saved")
                    self.uploadAudioSample()
                }
        })
    }

OPTION 2 - USE THE iWATCH's NATIVE SPEECH RECOGNITION In this approach I take the original, native voice menu, but...! I don't add any button options, just pure ASR. I launched the empty voice menu, and then recover the string returned by the ASR. Here's the code, enjoy:

func launchIWatchVoiceRecognition(){
     //you can see the empty array [], add options if it suits you well
        self.presentTextInputControllerWithSuggestions([], allowedInputMode: WKTextInputMode.Plain, completion:{(results) -> Void in
            let aResult = results?[0] as? String
            if(!(aResult == nil)){
                print(aResult) //print result
                self.sendMessageToPhone("Native ASR says:  "+aResult!)
                dispatch_async(dispatch_get_main_queue()) {
                    self.txtWatch.setText(aResult) //show result on UI
                }
            }//end if
        })//end show voice menu
    }

OPTION 2 is lightning fast, but OPTION 1 can be more handy if you want to do some advanced speech recon functions (custom vocabularies, grammar...) I would recommend OPTION 1 for most users. Voila!! If you need extra hints let me know!

Answer 3

I have tried to record audio directly from AppleWatch with AppleWatch Simulator.

But this only produce sound from iPhone simulator. You can do The same with real AppleWatch to record audio from iPhone microphone.

- (void) setupRecorder{
    [self configureAVAudioSession];

    // Set the audio file
    NSArray *pathComponents = [NSArray arrayWithObjects:
                               [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject], // NSUserDomainMask
                               @"AWAudio.wav",
                               nil];

    file = [NSURL fileURLWithPathComponents:pathComponents];

    // Setup audio session
    AVAudioSession *session = [AVAudioSession sharedInstance];
    [session setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];

    // Define the recorder setting
    NSMutableDictionary *recordSetting = [[NSMutableDictionary alloc] init];

    [recordSetting setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM] forKey:AVFormatIDKey];
    [recordSetting setValue:[NSNumber numberWithFloat:16000.0] forKey:AVSampleRateKey];
    [recordSetting setValue:[NSNumber numberWithInt: 1] forKey:AVNumberOfChannelsKey];
    [recordSetting setValue:[NSNumber numberWithBool:0] forKey:AVLinearPCMIsBigEndianKey];
    [recordSetting setValue:[NSNumber numberWithInt:16] forKey:AVLinearPCMBitDepthKey];

    // Initiate and prepare the recorder
    recorder = [[AVAudioRecorder alloc] initWithURL:file settings:recordSetting error:NULL];
    recorder.delegate = self;
    recorder.meteringEnabled = YES;

    [recorder prepareToRecord];
}

Example of how to record:

if (player.playing) {
        NSLog(@"player playing");
        [player stop];
    }

    if (!recorder.recording) {
        NSLog(@"recorder no recording");
        AVAudioSession *session = [AVAudioSession sharedInstance];
        [session setActive:YES error:nil];

        // Start recording
        [recorder record];
        [self startAnimation];
        [self performSelector:@selector(finishRecordingAfterTwelveSeconds) withObject:nil afterDelay:12.0];
    } else {

        // Pause recording
        AVAudioSession *audioSession = [AVAudioSession sharedInstance];
        [audioSession setActive:NO error:nil];
        [self stopAnimation];
        [self setupFinishRecording];
    }