4

I have tried doing POS tagging using openNLP POS Models on a normal Java application. Now I would like to implement it on Android platform. I am not sure what is the Android requirement or restrictions as I am not able to read the models (binary file) and execute the POS tagging properly.

I tried getting the .bin file from external storage as well as putting it in an external libraries but still it couldn't work. These are my codes:

InputStream modelIn = null;
POSModel model = null;

String path = Environment.getExternalStorageDirectory().getPath() + "/TextSumIt/en-pos-maxent.bin";

modelIn = new BufferedInputStream( new FileInputStream(path));
model = new POSModel(modelIn);

The error I got:

11-15 06:39:35.072: W/System.err(565): opennlp.tools.util.InvalidFormatException: The profile data stream has an invalid format!
11-15 06:39:35.177: W/System.err(565):  at opennlp.tools.dictionary.serializer.DictionarySerializer.create(DictionarySerializer.java:224)
11-15 06:39:35.177: W/System.err(565):  at opennlp.tools.postag.POSDictionary.create(POSDictionary.java:282)
11-15 06:39:35.182: W/System.err(565):  at opennlp.tools.postag.POSModel$POSDictionarySerializer.create(POSModel.java:48)
11-15 06:39:35.182: W/System.err(565):  at opennlp.tools.postag.POSModel$POSDictionarySerializer.create(POSModel.java:44)
11-15 06:39:35.182: W/System.err(565):  at opennlp.tools.util.model.BaseModel.<init>(BaseModel.java:135)
11-15 06:39:35.197: W/System.err(565):  at opennlp.tools.postag.POSModel.<init>(POSModel.java:93)
11-15 06:39:35.197: W/System.err(565):  at com.main.textsumit.SummarizationActivity.postagWords(SummarizationActivity.java:676)
11-15 06:39:35.205: W/System.err(565):  at com.main.textsumit.SummarizationActivity.generateSummary(SummarizationActivity.java:252)
11-15 06:39:35.205: W/System.err(565):  at com.main.textsumit.SummarizationActivity.onCreate(SummarizationActivity.java:127)

What is it that cause it not reading the model properly? And how should I resolve this? Please help.

Thank you.

Community
  • 1
  • 1
mellissa
  • 87
  • 1
  • 3
  • 13
  • Using AssetManager might help? Try moving the files into your project's assets folder. Then try the following: AssetManager am = getAssets(); InputStream is = am.open("en-pos-maxent.bin"); – Bjerva Nov 15 '12 at 08:27
  • i tried with AssetManager as well.. it is not reading the model as well. Any other way? Can I put into an external libraries and call from it? Is it possible? – mellissa Nov 15 '12 at 13:08

3 Answers3

2

@shivampip's solution kept giving me the following error:

java.io.FileNotFoundException: This file can not be opened as a file descriptor; it is probably compressed

All I had to do was add the following in my build.gradle:

android{
    aaptOptions {
        noCompress "bin"
    }
}

As a noob to Android I wish I saw this, so just adding this comment. @shivampip's solution worked like a charm though

Piyush Mundhra
  • 21
  • 4
1

For what it's worth, if this is still an issue: I had a similar issue attempting to use the POS model in a different context (non-Android), and in my case it appeared to be the extraction failing from the bin file, not anything with the model itself. It appears to be local to the tags.tagdict file in the archive (as suggested here http://sharpnlp.codeplex.com/discussions/263620), so if you don't need that currently (and I didn't for my simple scenarios) then try removing it from the archive. (But leave the archive intact as it's expected to arrive in zip'd form.)

Gene
  • 1,587
  • 4
  • 18
  • 38
  • Hi, thanks for this very useful insights. it does solve my problem. the only thing is that running this model on Android is a bit heavy. Am thinking of a way to run it somewhere else to process it. – mellissa Nov 26 '12 at 15:45
  • I could definitely see that being quite heavy on android, especially the load time if it's just for a quick recognition. To be honest I don't have much android experience myself, but I would think there must be a package for the native recognition that you could reuse for simple parsing. If you need the full OpenNlp POS, I'd think your best bet would be putting it on a server if it's an option. I'll let ya know if I come across anything in my travels though, best of luck! – Gene Nov 27 '12 at 16:21
  • Yes, I think opting for the server will be the best choice. But I need something somewhere to kickstart the developing as I am not familiar with running activities on server. if you have do share thanks :) – mellissa Nov 28 '12 at 01:43
1

Try this, it worked for me

    System.setProperty("org.xml.sax.driver", "org.xmlpull.v1.sax2.Driver");
    try {
        AssetFileDescriptor fileDescriptor = 
        context.getAssets().openFd("en_pos_maxent.bin");
        FileInputStream inputStream = fileDescriptor.createInputStream();
        POSModel posModel = new POSModel(inputStream);
        posTaggerME = new POSTaggerME(posModel);
    } catch (Exception e) {}
shivampip
  • 2,004
  • 19
  • 19