KenLM is a fast and low-memory language modeling toolkit that scales to trillions of words.
Questions tagged [kenlm]
23 questions
0
votes
1 answer
['kenlm/build/bin/build_binary', '-a', '255', '-q', '8', '-v', 'trie', 'lm_filtered.arpa', '/content/lm.binary']' returned non-zero exit status 1
During the build of lm binay to create scorer doe deepspeech model I was getting the following error again and again
subprocess.CalledProcessError: Command '['/content/kenlm/build/bin/build_binary', '-a', '255', '-q', '8', '-v', 'trie',…
Danish Bansal
- 608
- 1
- 7
- 25
0
votes
1 answer
Why does Kenlm lm model keep returning the same score for different words?
Why is the kenlm model returning the same values? I have tried it with a 4-gram arpa file as well. same issue.
import kenlm
model = kenlm.mode('lm/test.arpa') # unigram model.
print( [f'{x[0]:.2f}, {x[1]}, {x[2]}' for x in model.full_scores('this…
sourabh gupta
- 61
- 1
- 9
0
votes
3 answers
Set up kenlm for Windows
The official website makes it pretty clear that there is no support for kenlm in Windows. There is a Windows tag at the github repository but it seems to be maintained by few random contributors then and there.
How to set up kenlm for Windows then?
Bendemann
- 735
- 11
- 31
0
votes
1 answer
How can i use kenlm to check word alignment in a sentence?
I have seen many blogs saying language models can be used for numerous tasks but I cannot find any good implementations other than just the text generation.
My query is,
How can i use a language model like kenlm to correct my sentences for…
0
votes
1 answer
Compile bzip2 for Android?
I'm trying to crosscompile Kenlm for Android from ubuntu 18.04. To do that i need to compile all its dependecies first. I succeeded in compiling boost for android but I don't find how to do that whith bzip2.
When I use cmake to cross compile kenlm,…
user13583939
- 63
- 7
0
votes
1 answer
Kenlm language model scoring with Java on Windows
I'm mid-way through a Java project using a '.arpa' file to extract n-gram probabilities.
Ideally I would like to use a '.klm' file (created using the '.arpa' file), similar to:
model = kenlm.LanguageModel('languageModel.klm')
model.score('The dog…
Clare
- 5
- 8
0
votes
0 answers
Use anaconda 2 after anaconda 3 upgrade
I am trying to do a pip install within anaconda, i.e. with an environment activated. When I had previously installed the package with anaconda 2, it worked fine. When I do the same install with anaconda 3, I get the error reproduced below. Is there…
Adam_G
- 7,337
- 20
- 86
- 148
0
votes
0 answers
kenlm.Model' object has no attribute 'score' while running on google cloud ML
AttributeError: 'kenlm.Model' object has no attribute 'score'
>>> model = kenlm.Model('LM/en.europarl-nc.lm')
>>> model.score('This is a test')
Traceback (most recent call last):
File "", line 1, in
AttributeError:…
Appu
- 83
- 9