0

I am looking for a simple procedure to generate FST (finite state transducer) from cmudict-0.7b or cmudict-0.7b.dict, which will be used with phonetisaurus.

I tried following set of commands (phonetisaurus Aligner, Google NGramLibrary and phonetisaurus arpa2wfst) and able to generate FST but it didn't work. I am not sure where I did a mistake or miss any step. I guess very first command ie phonetisaurus-align, is not correct. 

phonetisaurus-align --input=cmudict.dict --ofile=cmudict/cmudict.corpus --seq1_del=false

ngramsymbols < cmudict/cmudict.corpus > cmudict/cmudict.syms

/usr/local/bin/farcompilestrings --symbols=cmudict/cmudict.syms --keep_symbols=1 cmudict/cmudict.corpus > cmudict/cmudict.far

ngramcount --order=8 cmudict/cmudict.far > cmudict/cmudict.cnts

ngrammake --v=2 --bins=3 --method=kneser_ney cmudict/cmudict.cnts > cmudict/cmudict.mod

ngramprint --ARPA cmudict/cmudict.mod > cmudict/cmudict.arpa

phonetisaurus-arpa2wfst-omega --lm=cmudict/cmudict.arpa > cmudict/cmudict.fst

I tried fst with phonetisaurus-g2p as follows:

phonetisaurus-g2p --model=cmudict/cmudict.fst --nbest=3 --input=HELLO --words

But it didn't return anything.... Appreciate any help on this matter.

g10dras
  • 399
  • 2
  • 11

1 Answers1

2

It is very important to keep dictionary in the right format. Phonetisaurus is very sensitive about that, it requires word and phonemes to be tab separated, spaces would not work then. It also does not allow pronunciation variant numbers CMUSphinx uses like (2) or (3). You need to cleanup dictionary with simple python script for example before feeding it into phonetisaurus. Here is the one I use:

#!/usr/bin/python

import sys

if len(sys.argv) != 3:
    print "Split the list on train and test sets"
    print
    print "Usage: traintest.py file split_count"
    exit()

infile = open(sys.argv[1], "r")
outtrain = open(sys.argv[1] + ".train", "w")
outtest = open(sys.argv[1] + ".test", "w")

cnt = 0
split_count = int(sys.argv[2])

for line in infile:
    items = line.split()
    if items[0][-1] == ')':
        items[0] = items[0][:-3]
    if items[0].find("_") > 0:
        continue
    line = items[0] + '\t' + " ".join(items[1:]) + '\n'
    if cnt % split_count == 3:
        outtest.write(line)
    else:
        outtrain.write(line)
    cnt = cnt + 1
Nikolay Shmyrev
  • 24,897
  • 5
  • 43
  • 87