Chaquopy Not able to download NLTK resource

Question

I am attempting to live download required packages for nltk using the chaquopy tool on android.

I am attempting to download the models into a specific data directory like so:

from android.os import Bundle
from android.support.v7.app import AppCompatActivity
from activity import R
from com.chaquo.python import Python
from java import jvoid, Override, static_proxy
import nltk
import os

class MainActivity(static_proxy(AppCompatActivity)):
    @Override(jvoid, [Bundle])
    def onCreate(self, state):
        AppCompatActivity.onCreate(self, state)
        self.setContentView(R.layout.activity_main)
        download_dir = "{}/nltk".format(Python.getPlatform().getApplication().getFilesDir())
        if not os.path.exists(download_dir):
            os.mkdir(download_dir)
        os.environ['NLTK_DATA'] = download_dir
        print('Download Dir: {}'.format(download_dir))
        #print(nltk.__version__)
        #dl = nltk.downloader.Downloader('http://nltk.org/nltk_data/')
        #dl.download('punkt', download_dir=download_dir)
        nltk.download('punkt')
        self.findViewById(R.id.text_box_1).setText(str(nltk.word_tokenize("Hello Python")))

I recieve the following error, as the client is unable to find or download the required package:

E/AndroidRuntime: FATAL EXCEPTION: main Process: process, PID: 22186 java.lang.RuntimeException: Unable to start activity ComponentInfo{/utils.Utils.MainActivity}: com.chaquo.python.PyException: LookupError:

  Resource [93mpunkt[0m not found.
  Please use the NLTK Downloader to obtain the resource:

  [31m>>> import nltk
  >>> nltk.download('punkt')
  [0m
  For more information see: https://www.nltk.org/data.html

  Attempted to load [93mtokenizers/punkt/PY3/english.pickle[0m

  Searched in:
    - '/usr/local/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - ''
**********************************************************************

    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3121)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3257)
    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:81)
    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1935)
    at android.os.Handler.dispatchMessage(Handler.java:107)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:7116)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:925)
 Caused by: com.chaquo.python.PyException: LookupError: 
**********************************************************************
  Resource [93mpunkt[0m not found.
  Please use the NLTK Downloader to obtain the resource:

  [31m>>> import nltk
  >>> nltk.download('punkt')
  [0m
  For more information see: https://www.nltk.org/data.html

  Attempted to load [93mtokenizers/punkt/PY3/english.pickle[0m

  Searched in:
    - '/usr/local/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - ''
**********************************************************************

    at <python>.nltk.data.find(data.py:701)
    at <python>.nltk.data._open(data.py:995)
    at <python>.nltk.data.load(data.py:870)
    at <python>.nltk.tokenize.sent_tokenize(__init__.py:104)
    at <python>.nltk.tokenize.word_tokenize(__init__.py:143)
    at <python>.utils.Utils.onCreate(Utils.py:23)
    at <python>.chaquopy_java.call(chaquopy_java.pyx:283)
    at <python>.chaquopy_java.Java_com_chaquo_python_PyObject_callAttrThrows(chaquopy_java.pyx:255)
    at com.chaquo.python.PyObject.callAttrThrows(Native Method)
    at com.chaquo.python.PyObject._chaquopyCall(PyObject.java:222)
    at utils.Utils.MainActivity.onCreate(MainActivity.java:24)
    at android.app.Activity.performCreate(Activity.java:7698)
    at android.app.Activity.performCreate(Activity.java:7687)
    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1299)
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3096)
        ... 11 more

I am aware of the thread: What directory does chaquopy code search for the Python packages which are imported in the Python code of the Android app code which seems to be a similar situation but does not solve my problem.

Answer 1

Try updating to Chaquopy 4.0.0 or newer. These versions set the HOME environment variable to your app's files directory, and NLTK will automatically create an nltk_data subdirectory there. Then there will no longer be any need to use the NLTK_DATA environment variable, or the download_dir parameter, and that code should all be removed.

(If you're already using Chaquopy 4.0.0 or newer, please add a comment with your versions of Chaquopy and NLTK.)

Because of an emulator bug, you may need to call nltk.download in a loop, as described in this answer.