Stanford Parser : frenchFactored.ser.gz

Question

I am using the Stanford Parser (Version 3.6.0) for French. My command line is

java -cp stanford-parser.jar:* edu.stanford.nlp.parser.lexparser.LexicalizedParser -maxlength 30 -outputFormat conll2007 frenchFactored.ser.gz test_french.txt > test_french.conll10

But I don't get the functions in the output, see :

1 Je _ CLS CLS _ 2 NULL _ _

2 mange _ V V _ 0 root _ _

3 des _ P P _ 2 NULL _ _

4 pommes _ N N _ 3 NULL _ _

5 . _ PUNC PUNC _ 2 NULL _ _

What could have I miss in the command line?

Answer 1

There's nothing wrong with your command:

Known formats are: oneline, penn, latexTree, xmlTree, words, wordsAndTags, rootSymbolOnly, dependencies, typedDependencies, typedDependenciesCollapsed, collocations, semanticGraph, conllStyleDependencies, conll2007. The last two are both tab-separated values formats. The latter has a lot more columns filled with underscores. [...]

Source: http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/trees/TreePrint.html

You can try another -outputFormat.

Answer 2

There is a deep learning based French dependency parser in Stanford CoreNLP 3.6.0.

Download Stanford CoreNLP 3.6.0 here:

http://stanfordnlp.github.io/CoreNLP/download.html

Also make sure to get the French models jar that is also available on that page.

And then run this command to use the French dependency parser, make sure to have the French models jar in your CLASSPATH:

java -Xmx6g -cp "*:stanford-corenlp-full-2015-12-09/*" edu.stanford.nlp.pipeline.StanfordCoreNLP -props StanfordCoreNLP-french.properties -file sample-french-document.txt -outputFormat text

Answer 3

Your query is good, but the Stanford parser doesn't support this yet (version 3.6.0).

The following code prints "false" when using the french model. The command you are using checks for this internally and quietly avoids the analysis when false.

System.out.println(
  LexicalizedParser
    .loadModel("frenchFactored.ser.gz")
    .treebankLanguagePack()
    .supportsGrammaticalStructures()
);

That's why I'm using the Malt parser (http://www.maltparser.org/).

If you like the following output:

1   Je      Je      C   CLS     null    2   suj     _   _
2   mange   mange   V   V       null    0   root    _   _
3   des     des     P   P       null    2   mod     _   _
4   pommes  pommes  N   N       null    3   obj     _   _
5   .       .       P   PUNC    null    2   mod     _   _

Then use the following code that generates it (you can't simply use the command line). I'm using both Stanford and Malt to accomplish this:

LexicalizedParser lexParser = LexicalizedParser.loadModel("frenchFactored.ser.gz");
TokenizerFactory<CoreLabel> tokenizerFactory = PTBTokenizer.factory(new CoreLabelTokenFactory(), "");
ConcurrentMaltParserModel parserModel = ConcurrentMaltParserService.initializeParserModel(new File("fremalt-1.7.mco"));

Tokenizer<CoreLabel> tok = tokenizerFactory.getTokenizer(new StringReader("Je mange des pommes."));
List<CoreLabel> rawWords2 = tok.tokenize();
Tree parse = lexParser.apply(rawWords2);

// The malt parser requires token in the MaltTab format (Connll).
// Instead of using the Stanford tagger, we could have used Melt or another parser.
String[] tokens = parse.taggedLabeledYield().stream()
    .map(word -> {
        CoreLabel w = (CoreLabel)word;
        String lemma = Morphology.lemmatizeStatic(new WordTag(w.word(), w.tag())).word();
        String tag = w.value();

        return String.join("\t", new String[]{
            String.valueOf(w.index()+1),
            w.word(),
            lemma != null ? lemma : w.word(), 
            tag != null ? String.valueOf(tag.charAt(0)) : "_",
            tag != null ? tag : "_"
        });
    })
    .toArray(String[]::new);

ConcurrentDependencyGraph graph = parserModel.parse(tokens);
System.out.println(graph);

From there, you can programmatically traverse the graph by using:

graph.nTokenNodes()

If you use Maven, just add the following dependencies to your pom:

<dependency>
    <groupId>org.maltparser</groupId>
    <artifactId>maltparser</artifactId>
    <version>1.8.1</version>
</dependency>
<dependency>
    <groupId>edu.stanford.nlp</groupId>
    <artifactId>stanford-corenlp</artifactId>
    <version>3.6.0</version>
</dependency>   

Bonus: the imports

import org.maltparser.concurrent.ConcurrentMaltParserModel;
import org.maltparser.concurrent.ConcurrentMaltParserService;
import org.maltparser.concurrent.graph.ConcurrentDependencyGraph;
import org.maltparser.concurrent.graph.ConcurrentDependencyNode;
import org.maltparser.core.exception.MaltChainedException;

import edu.stanford.nlp.ling.CoreLabel;
import edu.stanford.nlp.ling.WordTag;
import edu.stanford.nlp.parser.lexparser.LexicalizedParser;
import edu.stanford.nlp.process.CoreLabelTokenFactory;
import edu.stanford.nlp.process.Morphology;
import edu.stanford.nlp.process.PTBTokenizer;
import edu.stanford.nlp.process.Tokenizer;
import edu.stanford.nlp.process.TokenizerFactory;
import edu.stanford.nlp.trees.Tree;

Extra: fremalt-1.7.mco file

http://www.maltparser.org/mco/french_parser/fremalt.html