0

I want to know which letter is being read for lip sync animation. I have "onboundary" event that fires when every new word starts, having same event for each letter could solve the problem.

I tried to use set interval for each letter of current word, but it didn't work(

I'm using standard web speech api.

MAZ
  • 643
  • 5
  • 18
  • speech isn't about letters ... for instance ... "sh" which letter should be selected? – Bravo Aug 13 '21 at 10:42
  • @Bravo yes, you are right. There is something called "phoneme", it could be perfect to have phonemes. Letters are enough to create basic cut out animation, similar to south park. Actually there are just 3 mouth states, "A", "O" and closed. For low fidelity lip sync it's enough. – MAZ Aug 13 '21 at 10:50
  • @Bravo I found at Microsoft Azure service exactly what I need, but unfortunately it supports only English language https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-speech-synthesis-viseme?pivots=programming-language-javascript#get-viseme-events-with-the-speech-sdk – MAZ Aug 13 '21 at 10:52
  • 1
    You'd have to convert you utterance text to space separated characters, feed it to synthesis in a separate window, play it with volume set to 0, catch those boundary events and report them back to parent window with `postMessage`. – morganney Sep 17 '22 at 13:55

0 Answers0