7

I am working on a project and I need to get the root of a given word (stemming). As you know, the stemming algorithms that don't use a dictionary are not accurate. Also I tried the WordNet but it is not good for my project. I found phpmorphy project but it doesn't include API in Java.

At this time I am looking for a database or a text file of english words with their different forms. for example:

run running ran ... include including included ... ...

Thank you for your help or advise.

Majid Darabi
  • 731
  • 6
  • 15

1 Answers1

9

You could download LanguageTool (Disclaimer: I'm the maintainer), which comes with a binary file english.dict. The LanguageTool Wiki describes how to dump that file as a text file:

java -jar morfologik-tools-1.6.0-standalone.jar fsa_dump -x -d english.dict

For run, the file will contain this:

ran run VBD
run run NN
run run VB
run run VBN
run run VBP
running run VBG
runs run NNS
runs run VBZ

The first column is the inflected form, the second is the base form, and the third is the part-of-speech tag according to the (slightly extended) Penn Treebank tagset.

Daniel Naber
  • 1,594
  • 12
  • 19
  • Since '13 that tool seems to have changed significantly. Which would the same parameters today to achieve same output? I can't find the stemming parameter. Is it still there? Also, I see that the genitive case e.g. `orthography's` is recognized as ` orthography[orthography/NN,B-NP-singular|E-NP-singular]'['/POS]s[s/POS,]`. Is that correct? – Thalis K. Nov 06 '17 at 09:54
  • 3
    [The wiki](http://wiki.languagetool.org/developing-a-tagger-dictionary#toc2) has a command that works for me with LanguageTool 3.9: `java -cp languagetool.jar org.languagetool.tools.DictionaryExporter -i org/languagetool/resource/en/english.dict -info org/languagetool/resource/en/english.info -o dictionary.dump` – Daniel Naber Nov 06 '17 at 13:43