Visual Studio Speech Synthesis loop

Question

I know I'm trying to do something above my level for now, but I'm still trying my best, nevertheless.

Until now I was focused on c++->Code::Blocks, highschool level, but now I tried messing around in Visual Studio until I joined a project.

I have some problems with a loop (the program is based on system speech, system speech synthesis, speech recognizer and process start).

I have 2 cases :

1)Inputing the vocal command "hi" -> it responds back with "hi". 2)Inputting "hello" -> it responds with "opening google" & opens that speciffic webpage.

Well, if it would work as it is supposed to. If I input "hi", it responds with "hi"->[ "opening google"+opening the page ]->in a loop

If I input "hello" it goes straight to the loop [ "opening google"+opening the page] I tried some break points to see what am I doing there, but I'm hopeless.

You have the code here :

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Speech.Synthesis;
using System.Speech.Recognition;
using System.Diagnostics;

namespace Command
{
    public partial class Speech : Form
    {
        SpeechSynthesizer s = new SpeechSynthesizer();
        public Speech()
        {
            InitializeComponent();
            SpeechRecognitionEngine reco = new SpeechRecognitionEngine();
            Choices list = new Choices();
            list.Add(new string[] { "hi", "hello" });
            Grammar gm = new Grammar(new GrammarBuilder(list));

            try
            {
                reco.RequestRecognizerUpdate();
                reco.LoadGrammar(gm);
                reco.SpeechRecognized += Reco_SpeechRecognized;
                reco.SetInputToDefaultAudioDevice();
                reco.RecognizeAsync(RecognizeMode.Multiple);
            }
            catch { }
        }

        private void Reco_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
        {
            string a = e.Result.Text;

            switch (a)
            {
                case ("hello"):
                    {
                        s.SpeakAsync("opening google");
                        Process.Start("https://www.google.com");
                        break;
                    }
                case ("hi"):
                    {
                        s.SpeakAsync("hi");
                        break;
                    }
            }
        }
    }
}

Answer 1

Having tested this, I got similar behaviour to you. Upon saying either phrase, the program would sometimes repeat one or other of the responses.

Having run some tests, I believe what is happening is that the SpeechRecognitionEngine is not only listening to what you are saying, it's also listening to what the SpeechSynthesizer is playing.

To demonstrate, I changed the word list to add cat and dog, and changed the event handler as follows:

private void Reco_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
    string a = e.Result.Text;

    switch (a)
    {
        case ("cat"):
            {
                s.SpeakAsync("dog");
                break;
            }
        case ("dog"):
            {
                s.SpeakAsync("cat");
                break;
            }
    }
}

When I said "Cat", the system responded with "Dog". It then heard itself saying "Dog" and responded with "Cat", to which it responded "Dog". The end result being "Cat", "Dog", "Cat", "Dog" for a few iterations (typically three or four time in my tests). Conversely, when I commented out the calls to SpeakAsync(), I experienced no looping. I'm not sure what causes it to stop hearing itself and therefore stop looping around, it could be some internal limitation. Just to note I was using a headset with a microphone so it was not my microphone picking up what the speakers were playing, it was the DefaultAudioDevice picking up what SpeechSynthesizer was playing.

I would imagine it was possible to configure the SpeechRecognitionEngine to listen only to the Default Input Device (the microphone), or alternatively configure the SpeechSynthesizer to only output through the Default Output Device (the speakers) so that it does not cross the streams. However, the documentation on how you configure the audio streams is lacking in this regard.

Quite how "hi" sounds like "hello" or how "opening google" sounds like "hello" I'm not quite certain, but with such a limited grammar it may be that the recognition engine is being quite generous on what matches.

Hope this helps