0

I have a project with more than one file of python code. I have a file for model, one for data utility, one for training the model. I know how to submit a model with all the code is in one file. How can I indicate that T have more file in my project? Maybe something need to added in the setup.py file or __init__.py.

My directory looks like this:

setup.py
trainer/
  __init__.py
  task.py
  model/
     seq2seq.py
     model.py
  data_utli.py
Chris Martin
  • 30,334
  • 10
  • 78
  • 137
mosnoi ion
  • 123
  • 10

2 Answers2

1

You do not need to manually create your own package, though you're welcome to if you want.

There are two important steps in getting the package to work automatically:

  1. Create a proper python package
  2. Ensure your setup.py is correct.

In your case, the model subdirectory is causing issues. The quick fix is to move trainer/model/* to trainer/. Otherwise, you probably want to make model a proper sub-package by adding a (probably blank) __init__.py file in the model/ subdirectory.

Next, ensure your setup.py file is correctly specified. A sample script is provided in this documentation, repeated here for convenience:

from setuptools import find_packages
from setuptools import setup

setup(name='trainer',
      version='0.1',
      include_package_data=True,
      description='blah',
      packages=find_packages()
)

You can verify that it worked by running:

python setup.py sdist

That will create a dist subdirectory with a file trainer-0.1.tar.gz. Extracting the contents of that file shows that all of the files were correctly included:

$ cd dist
$ tar -xvf trainer-0.1.tgz
$ find trainer-0.1/
trainer-0.1/
trainer-0.1/setup.py
trainer-0.1/setup.cfg
trainer-0.1/trainer
trainer-0.1/trainer/data_util.py
trainer-0.1/trainer/task.py
trainer-0.1/trainer/__init__.py
trainer-0.1/trainer/model
trainer-0.1/trainer/model/__init__.py
trainer-0.1/trainer/model/model.py
trainer-0.1/trainer/model/seq2seq.py
trainer-0.1/PKG-INFO
trainer-0.1/trainer.egg-info
trainer-0.1/trainer.egg-info/dependency_links.txt
trainer-0.1/trainer.egg-info/PKG-INFO
trainer-0.1/trainer.egg-info/SOURCES.txt
trainer-0.1/trainer.egg-info/top_level.txt
rhaertel80
  • 8,254
  • 1
  • 31
  • 47
0

i found the answer in the ml cloud documentation https://cloud.google.com/ml-engine/docs/how-tos/packaging-trainer#to_use_the_gcloud_tool_to_use_an_existing_package_already_in_the_cloud

mosnoi ion
  • 123
  • 10