9

I am deploying a Google Cloud function from a source repository. I can get the Google Function to deploy fine as long as I don't make any reference in my main.py to any of my user-defined functions in the repository (i.e. scripts/functions in either package or package2 below other than main.py).

.
├── package
|   ├── __init__.py
|   ├── main.py
|   ├── requirements.txt
|   ├── script1.py
|   └── script2.py
├── package2
|   ├── __init__.py
|   ├── script3.py
|   └── script4.py
└── ...

I have 2 recent Stack Overflow questions regarding this function deploy here and here. They have collectively guided me towards a solution where main.py and requirements.txt are within a package and I deploy the Google Cloud Function with that package as the source. The problem now though is that I need to be able to access functions/scripts in other packages/scripts. If I include a statement like from package.script1 import foo in main.py I get an error like the one below when I deploy the function:

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function load error: Code in file main.py can't be loaded.
Did you list all required modules in requirements.txt?
Detailed stack trace: Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/wo
rker.py", line 211, in check_or_load_user_function
    _function_handler.load_user_function()
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/wo
rker.py", line 140, in load_user_function
    spec.loader.exec_module(main)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/user_code/main.py", line 4, in <module>
    from package.script1 import foo
ModuleNotFoundError: No module named 'package'

I am currently deploying this function with this google function deploy statement:

gcloud functions deploy NAME --source https://source.developers.google.com/projects/PROJECT_ID/repos/REPOSITORY_ID/moveable-aliases/BRANCH_NAME/paths/package/ --trigger-topic TOPIC

Moving main.py and requirements.txt back to the root directory doesn't work due to the issues in this stack overflow question.

I need a way to import user-defined packages/functions during a gcloud functions deploy ... statement. My main.py makes use of functions in package.script1.py, package.script2.py, package2.script3.py, and package2.script4.py. How can this be accomplished?

EDIT/UPDATED INFORMATION

I do have __init__.py files in each of the packages (package and package2), but they're currently blank/empty.

I have tried the following gcloud deploy... options:

  1. main.py and requirements.txt in the root directory (instead of the configuration as shown above/originally) with this deploy command:

    gcloud functions deploy NAME --source https://source.developers.google.com/projects/PROJECT_ID/repos/REPOSITORY_ID/moveable-aliases/BRANCH --trigger-topic TOPIC

    This produces the following error:

    ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function load error: File main.py that is expected to define function doesn't exist

  2. In an attempt to explicitly call out the root directory as source, I've also tried this with the same file configuration as in #1

    gcloud functions deploy NAME --source https://source.developers.google.com/projects/PROJECT_ID/repos/REPOSITORY_ID/moveable-aliases/BRANCH/paths/ --trigger-topic TOPIC

    It gives me the same error as #1:

    ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function load error: File main.py that is expected to define function doesn't exist

  3. The using the original file structure in this question (with main.py and requirements.txt in package), I have tried the deploy statement provided above. It resolves the main.py not found issue, but gives me the error as stated in the original question above

What else should I try?

Jed
  • 1,823
  • 4
  • 20
  • 52

1 Answers1

0

You should structure your app like this:

.
├── main.py
├── package
│   ├── __init__.py
│   ├── script1.py
│   └── script2.py
├── package2
│   ├── __init__.py
│   ├── script3.py
│   └── script4.py
└── requirements.txt

Then in main.py, you can do:

from package.script1 import foo
from package2.script2 import bar
# etc
Dustin Ingram
  • 20,502
  • 7
  • 59
  • 82
  • That is exactly how I had my project structured in [this post](https://stackoverflow.com/questions/52432324/google-cloud-function-function-load-error-file-main-py-that-is-expected-to-de), but Google Cloud had issues finding main.py – Jed Sep 26 '18 at 19:00
  • Even if you deploy from the same directory as `main.py`? What error does it produce? – Dustin Ingram Sep 26 '18 at 19:11
  • Are you able to deploy from your local filesystem rather than from a cloud source repository? – Dustin Ingram Sep 27 '18 at 03:02
  • Do you mean using a "ZIP upload"? There are 4 options on the GUI Google Cloud Functions "Create function" page: (1) Inline editor, (2) Zip upload, (3) Zip from Cloud Storage, & (4) Cloud Source repository – Jed Sep 27 '18 at 03:28
  • No, I mean directly with the `gcloud` command line tool, e.g.: `gcloud functions deploy NAME --runtime python37 --trigger-topic TOPIC` – Dustin Ingram Sep 27 '18 at 15:50
  • I tried to deploy my functions with the structure and command from @DustinIngram but it produces this error: `ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function failed on loading user code. Error message: File main.py is expected to contain a function named FOO` – lmaonuts Apr 30 '20 at 18:19
  • @lmaonuts Does your `main.py` file contain a function named `FOO`? – Dustin Ingram Apr 30 '20 at 18:57
  • @DustinIngram I have only imports in my `main.py`, no function declared. Funny enough today it worked just fine... – lmaonuts May 03 '20 at 09:54