How to connect appengine to vpc-connector

Question

I have a django project that I deploy to Cloud Run. It uses two GCP resources - a Postgres database, and a Redis MemoryStore instance for caching.

Redis instances can only be connected to with serverless VPC, an extra service with extra costs. So in order to minimize costs, I've set up a shared VPC and share the Redis instance between multiple of my projects (each with their own unique key prefix to prevent key clashes).

I have been successful in getting my Cloud Run containers to connect to Redis, but I haven't yet figured out how to get my appengine runners in the Cloud Build process to connect to redis. On an update of code, you always want Django to do a migrate first to apply database schema changes. This should also be followed by a purge of redis. Hence my need to provide the appengine runner access to my vpc-connector.

My cloudbuild.yml file (before I succeed to connect redis to the migrate/purge redis step):

steps:
  - id: "build image"
    name: "gcr.io/cloud-builders/docker"
    args: ["build", "-t", "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:$COMMIT_SHA", "."]

  - id: "push image"
    name: "gcr.io/cloud-builders/docker"
    args: ["push", "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:$COMMIT_SHA"]

  - id: "apply migrations, purge redis"
    name: "gcr.io/google-appengine/exec-wrapper"
    vpc_access_connector:
      name: projects/<my-project>/locations/us-central1/connectors/redis
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}:$COMMIT_SHA",
        "-s",
        "${PROJECT_ID}:${_DEPLOY_REGION}:${_DATABASE_INSTANCE}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME},REDISHOST=${_REDIS_HOST},REDISPORT=${_REDIS_PORT},BUILD_ID=$BUILD_ID,CLOUD_RUN_INSTANCE=1",
        # '--vpc-connector', 
        # 'projects/<my-project>/locations/us-central1/connectors/redis', # shared VPC connector belonging to host project <my-project> 
        "--",
        "python",
        "manage.py",
        "migrate_and_purge_redis", # I made a custom django command that does a migration, then purges redis. Saves having to boot up two separate appengine instances
      ]

  # Deploy container image to Cloud Run
  - id: "deploy"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args:
      - 'run'
      - 'deploy'
      - '${_SERVICE_NAME}'
      - '--image'
      - 'gcr.io/$PROJECT_ID/${_SERVICE_NAME}:$COMMIT_SHA'
      - '--platform=managed'
      - '--region=${_DEPLOY_REGION}'
      - '--vpc-connector' 
      - 'projects/<my-project>/locations/us-central1/connectors/redis' # shared VPC connector belonging to host project <my-project> 
      - '--set-env-vars' 
      - 'REDISHOST=${_REDIS_HOST},REDISPORT=${_REDIS_PORT},BUILD_ID=$BUILD_ID,CLOUD_RUN_INSTANCE=1'

images:
  - "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}:$COMMIT_SHA"

timeout: 1800s

Attempt 1

I tried adding in the same flag and variable for appengine that works for cloud run. Hence the middle step changed to this:

  - id: "apply migrations, purge redis"
    name: "gcr.io/google-appengine/exec-wrapper"
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}:$COMMIT_SHA",
        "-s",
        "${PROJECT_ID}:${_DEPLOY_REGION}:${_DATABASE_INSTANCE}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME},REDISHOST=${_REDIS_HOST},REDISPORT=${_REDIS_PORT},BUILD_ID=$BUILD_ID,CLOUD_RUN_INSTANCE=1",
        '--vpc-connector', 
        'projects/<my-project>/locations/us-central1/connectors/redis', # shared VPC connector belonging to host project <my-project> 
        "--",
        "python",
        "manage.py",
        "migrate_and_purge_redis",
      ]

This produced the following error:

Step #3 - "apply migrations, purge redis": Status: Downloaded newer image for gcr.io/google-appengine/exec-wrapper:latest
Step #3 - "apply migrations, purge redis": gcr.io/google-appengine/exec-wrapper:latest
Step #3 - "apply migrations, purge redis": Invalid option: --
Finished Step #3 - "apply migrations, purge redis"

So I don't quite understand the use of --, but it appears to signify a new line/command?

Attempt 2

Following advice from https://cloud.google.com/appengine/docs/standard/python3/connecting-vpc#configuring of how to modify an appengine's app.yaml file, I tried this next:

  - id: "apply migrations, purge redis"
    name: "gcr.io/google-appengine/exec-wrapper"
    vpc_access_connector:
      name: projects/<my-project>/locations/us-central1/connectors/redis
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}:$COMMIT_SHA",
        "-s",
        "${PROJECT_ID}:${_DEPLOY_REGION}:${_DATABASE_INSTANCE}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME},REDISHOST=${_REDIS_HOST},REDISPORT=${_REDIS_PORT},BUILD_ID=$BUILD_ID,CLOUD_RUN_INSTANCE=1",
        "--",
        "python",
        "manage.py",
        "migrate_and_purge_redis",
      ]

This generated the following error:

Your build failed to run: failed unmarshalling build config cloudbuild.yaml: unknown field "vpc_access_connector" in google.devtools.cloudbuild.v1.BuildStep

So while the vpc_access_connector: field appears to work in app.yaml files, it is not recognized in cloudbuild.yml files.

score 2 · Answer 1 · answered Mar 18 '22 at 13:55

VPC connectors doesn't exist for Cloud Build. There is another solution: private pool. The private pools are VM created for you on demand and on a network dedicated to this VMs. Of course, all of those things are managed by Google and you see nothing in your project, BUT, because the network and the VM are dedicated your project, Google Cloud can peer this network with your VPC and make the things working.

The bad side is, because the VM are created on demand for your build, it takes 30s to 60s of warmup (starting the VM) before executing your pipeline. Therefore, the test process is slower that the managed pools

score 0 · Answer 2 · answered Jun 18 '23 at 09:58

I know its late but hoping this would save somebody's time. This is what I have done as suggested here.

  - id: "make migrations"
    name: "gcr.io/google-appengine/exec-wrapper"
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}",
        "-s",
        "${PROJECT_ID}:${_REGION}:${_INSTANCE_NAME}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME}",
        "-e",
        "DJANGO_SETTINGS=dev",
        "-e",
        "INSTANCE_UNIX_SOCKET=/cloudsql/${PROJECT_ID}:${_REGION}:${_INSTANCE_NAME}",
        "--",
        "python",
        "manage.py",
        "makemigrations",
      ] 


substitutions:
  _INSTANCE_NAME: postgresql-server # The Cloud SQL instance name.

and then in my settings.py file

DB_STRING = f"postgres://{os.environ.get('POSTGRES_USER')}:{os.environ.get('POSTGRES_PASSWORD')}@/{os.environ.get('POSTGRES_DATABASE')}?host={os.environ['INSTANCE_UNIX_SOCKET']}"

DATABASES = {'default': dj_database_url.parse(DB_STRING)}

How to connect appengine to vpc-connector

Attempt 1

Attempt 2

2 Answers2