2

I am currently working on a project for the acumos AI challenge, and I am creating a simple Tensorflow program just to try uploading. The problem I am having is that acumos supports using python dependencies using Requirements, but I have yet to find a way to add .tf files (for the pretrained tensorflow model), or .txt files (for data).

The site allows me to upload my model without adding my pretrained tensorflow model as a dependency, but when I run it on a docker on a different computer I get this error... 

[2018-08-03 21:48:58 +0000] [11] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "runner.py", line 102, in load
    return build_app(self.parsed_args)
  File "runner.py", line 115, in build_app
    model = load_model(pargs.modeldir)  # refers to ./model dir in pwd. generated by helper script also in this dir
  File "/usr/local/lib/python3.6/site-packages/acumos/wrapped.py", line 42, in load_model
    model = _load_model(f)
  File "/usr/local/lib/python3.6/site-packages/dill/dill.py", line 288, in load
    obj = pik.load()
  File "/usr/local/lib/python3.6/site-packages/dill/dill.py", line 546, in _load_type
    return _reverse_typemap[name]
KeyError: 'ClassType'
[2018-08-03 21:48:58 +0000] [11] [INFO] Worker exiting (pid: 11)
[2018-08-03 21:48:58 +0000] [10] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "runner.py", line 102, in load
    return build_app(self.parsed_args)
  File "runner.py", line 115, in build_app
    model = load_model(pargs.modeldir)  # refers to ./model dir in pwd. generated by helper script also in this dir
  File "/usr/local/lib/python3.6/site-packages/acumos/wrapped.py", line 42, in load_model
    model = _load_model(f)
  File "/usr/local/lib/python3.6/site-packages/dill/dill.py", line 288, in load
    obj = pik.load()
  File "/usr/local/lib/python3.6/site-packages/dill/dill.py", line 546, in _load_type
    return _reverse_typemap[name]
KeyError: 'ClassType'
[2018-08-03 21:48:58 +0000] [10] [INFO] Worker exiting (pid: 10)
[2018-08-03 21:48:58 +0000] [1] [INFO] Shutting down: Master
[2018-08-03 21:48:58 +0000] [1] [INFO] Reason: Worker failed to boot.

It is just my guess that this error is cause by introception missing the .tf files, but I could be totally wrong on that. I should also mention that my docker container works fine for models that I built that do not require external files (e.i. a simple a+b = c type model).

Any help on this would be great! Thanks!

Andrew James
  • 372
  • 1
  • 8

1 Answers1

0

Acumos doesn’t directly support adding arbitrary file type dependencies, and it won’t infer them automatically, unfortunately. The Requirements mechanism is there for Python dependencies. There are other ways to build pretrained Tensorflow models without using external files; see some of the models in the Acumos Open Source repository (https://gerrit.acumos.org). If you load your model (and the contents of any external files, like .txt) into memory before dumping or pushing, the Python Acumos Client will properly serialize it, and that is the typical usage pattern. See the Tensorflow example here: https://github.com/acumos/acumos-python-client/blob/master/examples/tensorflow_example.py. Instead of building the model from scratch, you can just deserialize the Tensorflow model. Then the rest of the example should still hold. . Another thing you might try is creating a package in a subdirectory that contains the files you want, and including that using the standard Requirements mechanism. I believe the Acumos client simply packages up everything you have in the package directory when creating the archive representing your model. But try the other suggestions first.

Mukesh Mantan
  • 16
  • Thanks for your help! I found this to be the case. Training it, and serializing it, then uploading it to acumos in one python script, or over a couple scripts seems to be the intended use case for acumos. – Andrew James Aug 04 '18 at 16:31