Can't find dependencies when deploying function in google cloud build

Question

So im trying to create a google cloud function that imports a python package called pdftotext. Now in order to pip install pdftotext you have to install some system dependencies. i.e:

sudo apt install build-essential libpoppler-cpp-dev pkg-config python3-dev

Now my solution to doing that is to create a requirements.txt and a cloudbuild.yml file that I upload to google source repositories and then use a cloud build trigger that listens to the repo, and deploys the function when something is pushed to the repo.

my cloudbuild.yml file looks like this:

 steps:
  # Install OS Dependencies
  - name: "docker.io/library/python:3.9"
    id: "OS Dependencies"
    entrypoint: bash
    args:
      - '-c'
      - | 
        apt-get update
        apt-get install -y build-essential libpoppler-cpp-dev pkg-config python3-dev
        apt-get install -y pip
        pip3 install -t /workspace/lib -r requirements.txt
  # Deploy Function      
  - name: "gcr.io/cloud-builders/gcloud"
    id: "Deploy Function"
    args:
      [ 
        "functions",
        "deploy",
        "pdf_handler",
        "--entry-point",
        "main",
        "--source",
        ".",
        "--runtime",
        "python39",
        "--memory",
        "256MB",
        "--service-account",
        "my_service_account",
        "--trigger-http",
        "--timeout",
        "540",
        "--region",
        "europe-west1",
      ]
options:
  logging: CLOUD_LOGGING_ONLY

The trigger tries to deploy the function but i keep getting this error even though i installed the OS dependencies

"Deploy Function": pdftotext.cpp:3:10: fatal error: poppler/cpp/poppler-document.h: No such file or directory

It seems like the function deployment can't find the location where the dependencies are installed.

I've tried installing and deploying in the same step but still get the same error.

Any advice is appreciated.

Thanks in advance!

Answer 1

When you deploy with Cloud Functions, ONLY your code is taken and packaged (in a container) by the service.

During the packaging, another Cloud Build is called to build that container (with Buildpacks.io) and then to deploy it. That deployment doesn't care that you install some APT packages in your environment. But your /lib directory is uploaded to that new Cloud Build

You should update your requirements.txt of the Cloud Functions code that you deploy to point to the /lib directory to prevent PIP looking for external package (and compilation requirement)