14

I want to develop a Windows application on Windows 10 using the new Cortana engine.

Unfortunately as far as I know, it seems to be available only on Windows Phone 8.1 project (for instance, I didn't find a way to access to the Windows.Media.SpeechRecognition namespace from a different type of Visual Studio project).

Also I wasn't able to find a good API documentation, only some very simple examples.

Edit:

Based on Peter Torr answer I've wrote some code. I've been able to recognize some word but the engine seems to struggle when it tried to recognize some simple words like "Hello", while Cortana recognized it successfully.

Am I doing something wrong?

public static class SpeechSynthetizerManager
{
    private static readonly SpeechSynthesizer synth = new SpeechSynthesizer();
    private static readonly SpeechRecognitionEngine speechRecognitionEngine = new SpeechRecognitionEngine();

    public static event EventHandler<SpeechRecognizedEventArgs> SpeechRecognized
    {
        add { speechRecognitionEngine.SpeechRecognized += value; }
        remove { speechRecognitionEngine.SpeechRecognized -= value; }
    }

    public static event EventHandler<RecognizeCompletedEventArgs> RecognizeCompleted
    {
        add { speechRecognitionEngine.RecognizeCompleted += value; }
        remove { speechRecognitionEngine.RecognizeCompleted -= value; }
    }

    static SpeechSynthetizerManager()
    {
        synth.SelectVoiceByHints(VoiceGender.Female);

        speechRecognitionEngine.LoadGrammar(new DictationGrammar());

        speechRecognitionEngine.SetInputToDefaultAudioDevice();
    }

    public static void Speak(string message)
    {
        synth.Speak(message);
    }

    public static void Listen()
    {
        speechRecognitionEngine.RecognizeAsync();
    }
}
simoneL
  • 602
  • 1
  • 7
  • 23

3 Answers3

21

Strictly speaking, the Cortana APIs are the ones in the Windows.ApplicationModel.VoiceCommands namespace. These are not available to Classic ("Desktop") apps, but are available to Universal Windows apps on Windows 10. The reason Classic apps can't use the APIs is because they rely on concepts such as Background Tasks and App Identity that don't apply to Classic apps.

The types in the Windows.Media.SpeechRecognition namespace are also unavailable to Classic apps, but I'm not sure what the limitation is there.

Note: As @Andrew Pilley mentions, you might be able to get these types to work in a Desktop app but that's not explicitly supported at the moment.

If you just want speech recognition in a .NET app, you can use the System.Speech.Recognition namespace, which uses the same underlying technology.

Community
  • 1
  • 1
Peter Torr - MSFT
  • 11,824
  • 3
  • 18
  • 51
  • Hi, thank you very much for your reply. I've tried to create a class library project but in the list of available references there is no Windows.Media (or similar) assembly. The only DLLs with that name exist in path like C:\Windows\WinSxS\amd64_microsoft-windows-runtime-windows-media_31bf3856ad364e35_10.0.10240.16384_none_4f69ddc0692a98bf and they are COM dlls. Honestly, I feel a bit confused. I would have expect the dll to be part of .Net Framework (or windows Sdk) and to be a C# library. – simoneL Aug 10 '15 at 23:28
  • I think you missed this part of the answer: "The types in the Windows.Media.SpeechRecognition namespace are also unavailable to Classic apps". I think you need to be looking at System.Speech.Recognition. – Brendan Green Aug 11 '15 at 00:38
  • Yes Brendan, you're right, I've misread. Next I should go to bed earlier :) By the way, I've wrote some code based on Peter reply and included it into the original question. I achieved some speech recognition, but it's still not good enough. – simoneL Aug 11 '15 at 21:18
  • I ended up developing a new Windows Universal App and it seems is using the Cortana speech recognition engine in the end. This is the sample that I followed: https://www.projectoxford.ai/doc/speech/Get-Started/windows – simoneL Aug 12 '15 at 22:06
5

So, while Peter Torr is right about Cortana (the Windows.ApplicationModel.VoiceCommands API), the new Speech Recognition WinRT APIs (Windows.Media.SpeechRecognition) can be used in a classic C# app, if you're willing to use the WinRT Interop libraries.

I've detailed how to set that up in this answer to a stack overflow question, although the person who asked that question is having some trouble replicating what I've gotten to work locally.

Community
  • 1
  • 1
Andrew Pilley
  • 824
  • 6
  • 9
  • Oh that looks promising! I'll try to implement it as soon as I can :) – simoneL Aug 13 '15 at 06:11
  • 1
    Be careful - the APIs might "work" but they are not explicitly tested for Desktop use (it doesn't have the `DualApiPartitionAttribute` or `MuseAttribute` in the WinMD). – Peter Torr - MSFT Aug 16 '15 at 06:08
0

You can activate Cortana's "WIN+C" event by just calling this "uri" like a new Process(), even when the setting is turned off.

ms-cortana://StartMode=Reactive&ListeningMode=True&QuerySource=VoiceShortcutKey&Positioner=desktop&SpeechTextInputHeight=40&ColorPrevalenceEnabled=true&PersonaOnlySupported=true&TaskbarOrientation=3&SearchBoxInTaskbar=false&AppsUseLightTheme=false&HideFileExtensions=false