4

I have a mapping of 100,000+ words to their phonemes (CMUdict), like:

ABANDONED => [ 'AH', 'B', 'AE', 'N', 'D', 'AH', 'N', 'D' ]

I want to split the original words' letters into a number of groups equal to the number of phonemes, e.x.

ABANDONED => [ 'A', 'B', 'A', 'N', 'D', 'O', 'N', 'ED' ]

I don't have a mapping of phonemes to graphemes, but it seems like I should be able to compute a statistical model of phonemes to graphemes, then use that to decide where to split each word. (It would be nice if the model could also be used to convert new words to their probable phonemes)

How can I do this? I was thinking a hidden Markov model sounds like it could be applicable, but beyond that hunch I don't know.

alvas
  • 115,346
  • 109
  • 446
  • 738
tlrobinson
  • 2,812
  • 1
  • 33
  • 35
  • This seems to be a related question: https://stackoverflow.com/questions/10825685/chunking-english-words-into-graphemes-corresponding-to-distinct-sounds – Ivan Sysoev Mar 11 '18 at 23:56

1 Answers1

1

To gather statistics, first align the word to its phonetic representation by matching the identical letters and phonemes (like N and N). You can get the best match with dynamic programming. Then you can map the remaining characters of the words to the remaining phonemes.

Once you calculate the frequencies, you can use the noisy channel model to convert new words to phonemes.

Vsevolod Dyomkin
  • 9,343
  • 2
  • 31
  • 36