Where is the stream number in the event info for System.Speech.Synthesis?

Question

One can let the SpeechSynthesizer speak text in an asynchronous way, for example like this:

Private WithEvents _Synth As New SpeechSynthesizer

Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
    If e.KeyCode = Keys.Enter Then
        _Synth.SpeakAsync(New Prompt(Me.TextBox1.Text))
    End If
End Sub

The events that SpeechSynthesizer generates enables us to tell what the computer voice is just speaking.

For example, you may visualize the speech output by selecting the characters like this:

Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress

    Me.TextBox1.SelectionStart = e.CharacterPosition
    Me.TextBox1.SelectionLength = e.CharacterCount

End Sub

However, when SpeakAsync is called repeatedly (for example when we tell the SpeechSyntesizer to speak the same text while it's currently just speaking), the speech requests are queued, and the SpeechSynthesizer plays them one by one.

However, I haven't been able to find out which request the synthesizer is currently speaking. The SpeakProgressEventArgs don't reveal this:

Using SAPI5, the events provided a StreamNumber:

Parameters
StreamNumber
    The stream number which generated the event. When a voice enqueues more than one stream by speaking asynchronously, the stream number is necessary to associate an event with the appropriate stream.

Using this StreamNumber, you could always tell what the SpeechSynthesizer is just playing / speaking.

The System.Speech.Synthesis implementation is a modern version of the SAPI5 implementation.

However, I just don't find a StreamNumber indiciator or similiar information.

System.Speech.Synthesis provides information about just everything that is just happening, so it's highly unlikely that it doesn't provide the information which of the requests it's just processing.

How could this be retrieved?

Answer 1

To clarify my comment about using the Prompt Class to hold any identifying state you need, consider the following where the Prompt holds a reference to the source TextBox.

Imports System.Speech.Synthesis
Public Class MyPrompt : Inherits Prompt
    Private tbRef As WeakReference(Of TextBox)

    Public Sub New(textBox As TextBox)
        MyBase.New(textBox.Text)
        ' only hold a weak reference to the TextBox
        ' to avoid any disposal issues
        tbRef = New WeakReference(Of TextBox)(textBox)
    End Sub

    Public ReadOnly Property SourceTextBox As TextBox
        Get
            Dim ret As TextBox = Nothing
            tbRef.TryGetTarget(ret)
            Return ret
        End Get
    End Property
End Class

Now your original code could be written as:

Imports System.Speech.Synthesis

Public Class Form1
    Private WithEvents _Synth As New SpeechSynthesizer

    Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
        If e.KeyCode = Keys.Enter Then
            ' use a custom prompt to store the TextBox
            _Synth.SpeakAsync(New MyPrompt(Me.TextBox1))
        End If
    End Sub

    Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress
        Dim mp As MyPrompt = TryCast(e.Prompt, MyPrompt)
        If mp IsNot Nothing Then
            Dim tb As TextBox = mp.SourceTextBox
            If tb IsNot Nothing Then
                ' set the selection in the source TextBox
                tb.SelectionStart = e.CharacterPosition
                tb.SelectionLength = e.CharacterCount
            End If
        End If
    End Sub

End Class

---

Edit:

The OP wants to use this with the SpeakSsmlAsync method. That in itself is not possible as that method creates a base Prompt using the Prompt(String, SynthesisTextFormat) Constructor and returns the created Prompt after calling SpeechSynthesizer.SpeakAsync(created_prompt).

Below is a derived Prompt class that accepts either a string of ssml or a PromptBuilder instance along with an integer identifier. A new version of MyPrompt to use ssml and an integer identifer.

Imports System.Speech.Synthesis

Public Class MyPromptV2 : Inherits Prompt
    Public Sub New(ssml As String, identifier As Int32)
        MyBase.New(ssml, SynthesisTextFormat.Ssml)
        Me.Identifier = identifier
    End Sub

    Public Sub New(builder As PromptBuilder, identifier As Int32)
        MyBase.New(builder)
        Me.Identifier = identifier
    End Sub

    Public ReadOnly Property Identifier As Int32
End Class

...

Imports System.Speech.Synthesis

Public Class Form1
    Private WithEvents _Synth As New SpeechSynthesizer

    Private Sub TextBox1_KeyUp(sender As Object, e As KeyEventArgs) Handles TextBox1.KeyUp
        If e.KeyCode = Keys.Enter Then
            ' build some ssml from the text
            Dim pb As New PromptBuilder
            pb.AppendText(TextBox1.Text)
            ' use ssml and and integer
            _Synth.SpeakAsync(New MyPrompt(pb.ToXml, 10))
            ' or 
            '_Synth.SpeakAsync(New MyPrompt(pb, 10))
        End If
    End Sub

    Private Sub _Synth_SpeakProgress(sender As Object, e As SpeakProgressEventArgs) Handles _Synth.SpeakProgress
        Dim mp As MyPromptV2 = TryCast(e.Prompt, MyPromptV2)
        If mp IsNot Nothing Then
            Select Case mp.Identifier
                Case 10
                    TextBox1.SelectionStart = e.CharacterPosition
                    TextBox1.SelectionLength = e.CharacterCount
            End Select
        End If
    End Sub
End Class

Answer 2

There is one another way to get which sentence is currently processing. You could assign choice numbers to your sentences and then you can recognize the speech by getting the index of that sentence; which you can further process conditions. Use SpeechRecognizedEventArgs argument of SpeechRecognized method for getting the sentence index.

void sre_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
  string txt = e.Result.Text;
  int sentenceIndex = txt.IndexOf("My Sentence");

  if (sentenceIndex >= 0)
  {
    Console.WriteLine("Currently Speaking Sentence: My Sentence, with index number: " 
                 + sentenceIndex);
  }

  //.... some code here
}

Follow full example here.

---

Edit 1:

The class-scope SpeechSynthesizer object gives the application the ability to speak. The SpeechRecognitionEngine object allows the application to listen for and recognize spoken words or phrases.