edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException when using CoreNLP with Jython

Question

I'm trying to use CoreNLP run off of Jython 2.7.1, ran using IntelliJ IDEA, with the following test code:

from edu.stanford.nlp.simple import *
s = Sentence("This is a test.")
ss = s.nerTags()
print(s)
print(ss)

Log:

Connected to pydev debugger (build 172.4155.5)
[MainThread] INFO edu.stanford.nlp.tagger.maxent.MaxentTagger - Loading POS tagger from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [0.4 sec].
[MainThread] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [0.8 sec].
[MainThread] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [0.4 sec].
[MainThread] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [0.4 sec].
[MainThread] INFO edu.stanford.nlp.time.JollyDayHolidays - Initializing JollyDayHoliday for SUTime from classpath edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.
Traceback (most recent call last):
  File "C:\Users\user\.IntelliJIdea2017.2\config\plugins\python\helpers\pydev\pydevd.py", line 1599, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Users\user\.IntelliJIdea2017.2\config\plugins\python\helpers\pydev\pydevd.py", line 1026, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:/Users/user/IdeaProjects/nlptest/src/test.py", line 3, in <module>
    ss = s.nerTags()
    at edu.stanford.nlp.util.ReflectionLoading.loadByReflection(ReflectionLoading.java:40)
    at edu.stanford.nlp.time.TimeExpressionExtractorFactory.create(TimeExpressionExtractorFactory.java:57)
    at edu.stanford.nlp.time.TimeExpressionExtractorFactory.createExtractor(TimeExpressionExtractorFactory.java:38)
    at edu.stanford.nlp.ie.regexp.NumberSequenceClassifier.<init>(NumberSequenceClassifier.java:86)
    at edu.stanford.nlp.ie.NERClassifierCombiner.<init>(NERClassifierCombiner.java:136)
    at edu.stanford.nlp.pipeline.NERCombinerAnnotator.<init>(NERCombinerAnnotator.java:91)
    at edu.stanford.nlp.pipeline.AnnotatorImplementations.ner(AnnotatorImplementations.java:70)
    at edu.stanford.nlp.simple.Document$2.get(Document.java:115)
    at edu.stanford.nlp.simple.Document$2.get(Document.java:109)
    at edu.stanford.nlp.simple.Document.runNER(Document.java:886)
    at edu.stanford.nlp.simple.Sentence.nerTags(Sentence.java:528)
    at edu.stanford.nlp.simple.Sentence.nerTags(Sentence.java:536)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException: edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException: Error creating edu.stanford.nlp.time.TimeExpressionExtractorImpl

I added, using my IDE, the CoreNLP directory to my CLASSPATH and it seems that Jython can find it. There are several similar questions on SO on about error but none of their solutions (if they had any) actually worked for me. Has anyone else found a solution to this problem? (For the record, CoreNLP works fine when run off of the command line.)

Edit (my attempt using Bash on Windows):

user@user:~/uh$ export JYTHONPATH="/home/user/uh/stanford-corenlp-full-2017-06-09/*:"    
user@user:~/uh$ jython
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0_131
Type "help", "copyright", "credits" or "license" for more information.
>>> from edu.stanford.nlp.simple import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named edu
>>>

Cameron M. · Accepted Answer · 2017-09-27T00:06:02.813

My further testing shows that this is purely a Jython problem and a cross-platform one at that. I tested similar setups on Windows, Ubuntu, and Mac resulting in the same error. Even running pure Java code designed to return the initial pipeline to Jython returns the same error. However, the error can be avoided by setting "ner.useSUTime" to false with the following:

props.setProperty("ner.useSUTime", "0")

score 0 · Answer 2 · answered Sep 03 '17 at 18:40

I'm not sure, what's going on. What version of CoreNLP are you using? Working at the command-line, I couldn't reproduce this problem. See below. So, my first two questions are: Are you using the latest CoreNLP? (If not, try upgrading.) Does this work for you at the command-line? (In which case, it's only the use in IntelliJ that is problematic - not sure what would be going on there but something to do with how reflection and classloaders work?)

==> Pouring jython-2.7.1.sierra.bottle.tar.gz
  /usr/local/Cellar/jython/2.7.1: 3,797 files, 147.4MB
$ export JYTHONPATH="/Users/manning/Software/stanford-corenlp-full-2017-06-09/*:"
$ jython
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_65
Type "help", "copyright", "credits" or "license" for more information.
>>> from edu.stanford.nlp.simple import *
>>> s = Sentence("This is a test.")
>>> ss = s.nerTags()
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/manning/git/javanlp/projects/core/lib/slf4j-simple.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/manning/Software/stanford-corenlp-full-2017-06-09/slf4j-simple.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
[main] INFO edu.stanford.nlp.tagger.maxent.MaxentTagger - Loading POS tagger from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [0.7 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [0.9 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [0.6 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [2.4 sec].
[main] INFO edu.stanford.nlp.time.JollyDayHolidays - Initializing JollyDayHoliday for SUTime from classpath edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.
>>> print(s)
This is a test.
>>> print(ss)
[O, O, O, O, O]
>>> s2 = Sentence("Chris joined Stanford in 1999");
>>> ss2 = s2.nerTags()
>>> print(ss2)
[PERSON, O, ORGANIZATION, O, DATE]
>>>

I gave up trying to run this purely on Windows since the behavior of Jython was honestly astounding (it was trying to pull files from the individual jar files from the local file system instead) and I had to use 'sys.path.append' on every jar file (in a similar fashion to how IntelliJ does it) to every get it to start but with similar failures. I instead tried to run it off of Bash on Windows and it now does not recognize JYTHONPATH at all (see above). — Cameron M., Sep 04 '17 at 10:01

edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException when using CoreNLP with Jython

2 Answers2