Use Kinect Voice Recognition to run Metro GUI application

Question

i want to use kinect sdk voice recognition to run application from Metro UI. For Example when i say the word:News, it would run from Metro UI the application of News.

Thanks to everybody!

Regards!

Answer 1

First you need to Make the connection with the audio stream and start listening:

    private KinectAudioSource source;
    private SpeechRecognitionEngine sre;
    private Stream stream;

private void CaptureAudio()
        {

            this.source = KinectSensor.KinectSensors[0].AudioSource;
            this.source.AutomaticGainControlEnabled = false;
            this.source.EchoCancellationMode = EchoCancellationMode.CancellationOnly;
            this.source.BeamAngleMode = BeamAngleMode.Adaptive;

        RecognizerInfo info = SpeechRecognitionEngine.InstalledRecognizers()
            .Where(r => r.Culture.TwoLetterISOLanguageName.Equals("en"))
            .FirstOrDefault();

        if (info == null) { return; }
        this.sre = new SpeechRecognitionEngine(info.Id);

        if(!isInitialized) CreateDefaultGrammars();

        sre.LoadGrammar(CreateGrammars()); //Important step

        this.sre.SpeechRecognized +=
            new EventHandler<SpeechRecognizedEventArgs>
                (sre_SpeechRecognized);
        this.sre.SpeechHypothesized +=
            new EventHandler<SpeechHypothesizedEventArgs>
                (sre_SpeechHypothesized);
        this.stream = this.source.Start();
        this.sre.SetInputToAudioStream(this.stream, new SpeechAudioFormatInfo(
            EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));
        this.sre.RecognizeAsync(RecognizeMode.Multiple);

    }

First you can see in the sample that there's one important step sre.LoadGrammar(CreateGrammars()); which creates and loads the grammar so you have to create the method CreateGrammars():

private Grammar CreateGrammars()
        {

        var KLgb = new GrammarBuilder();
        KLgb.Culture = sre.RecognizerInfo.Culture;
        KLgb.Append("News");
        return Grammar(KLgb);

    }

The above sample create a grammar listening for the word "News". Once it is recognized (the probability that the word said is one in your grammar is higher than a threshold), the speech recognizer engine (sre) raise the SpeechRecognized event.

Of course you need to add the proper handler for the two events (Hypothetize, Recognize):

    private void sre_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
    {

        this.RecognizedWord = e.Result.Text;
        if (e.Result.Confidence > 0.65) InterpretCommand(e);
    }

Know all you have to do is to write the InterpretCommand method which does whatever you want (as running a metro app ;) ). If you have multiple words in a dictionnary the method has to parse the text recognized and verify that this is the word news wich was recognized.

Here you can download samples of a great book on Kinect: Beginning Kinect Programming with the Microsoft Kinect SDK (unfortunately the book itself is not free). On the folder Chapter7\PutThatThereComplete\ you have a sample using audio that you can be inspired by.