Unable to find/modify the Dockerfile of a Google App Engine Managed VM that uses a standard runtime (python27)

Question

I want to modify the Dockerfile of a Google App Engine managed VM that uses a standard runtime (python27).

I want to do this to add a C++ library that needs to be called to implement an HTTP request. This library is pretty much the only addition I need to the sandboxed python27 runtime.

The documentation makes it quite clear that this is possible:

Each standard runtime uses a default Dockerfile, which is supplied by the SDK. You can extend and enhance a standard runtime by adding new docker commands to this file.

Elsewhere they say that they Dockerfile of a standard runtime will be generated in the project directory:

When you use gcloud to run or deploy a managed VM application based on a standard runtime (in this case Python27), the SDK will create a minimal Dockerfile using the standard runtime as a base image. You'll find this Dockerfile in your project directory...

This is the one I am supposed to modify according to the same page:

Later steps in this tutorial will show you how to extend the capabilities of your runtime environment by adding instructions to the Dockerfile.

The problem is that when I do run my application on the dev server, I cannot find the Dockerfile anywhere, so I can't make any changes to it.

Has anyone managed to modify the standard runtime Dockerfile for Google App Engine? Any help would be appreciated.

Answer 1

To use google-api-python-client i had the same issue, cause I needed pycrypto. I always got the error:

CryptoUnavailableError: No crypto library available

To solve this I created an instance start handler that installs all needed libs. It's ugly but it works.

app.yaml:

handlers:
- url: /_ah/start
  script: start_handler.app

start_handler.py

import webapp2
import logging
import os

class StartHandler(webapp2.RequestHandler):
  def execute(self, cmd):
    logging.info(os.popen("%s 2>&1" % cmd).read())

  def get(self):
    if not os.environ.get('SERVER_SOFTWARE','').startswith('Development'):
      self.execute("apt-get update")
      self.execute("apt-get -y install build-essential libssl-dev libffi-dev python-dev")
      self.execute("pip install cryptography")
      self.execute("pip install pyopenssl")


app = webapp2.WSGIApplication([
                                ('/_ah/start', StartHandler)
                              ], debug=True)

Answer 2

It seems like the Dockerfile is generated only when using gcloud preview app run and not dev_appserver.py, which was what I was using.

I am however not able to modify the Dockerfile and run a custom managed VM. But that is a seperate error (--custom_entrypoint related).

This whole situation is a nightmare fueled by atrocious documentation and support. A warning for other developers considering Google App Engine.

Answer 3

Turns out, extending the Dockerfile in your app does not work the way it's purported in the Documentation (Link). In fact, if there is a Dockerfile present you will get the following error:

"ERROR: (gcloud.preview.app.deploy) There is a Dockerfile in the current directory, and the runtime field in /[...]/app.yaml is currently set to [runtime: python27]. To use your Dockerfile to build a custom runtime, set the runtime field in [...]/app.yaml to [runtime: custom]. To continue using the [python27] runtime, please omit the Dockerfile from this directory"

The only way I've been able to use a customized Dockerfile is using a custom runtime.

Google has a really good GitHub example for deploying Django to a managed VM using a custom Python runtime (here).

Since you're using the custom runtime you'll have to implement health checking yourself. However, if you need to access Google APIs, Google has an example of how to set that up on GitHub (here).

For help implementing health checking, or integrating with Google APIs you can follow the Google Compute Engine, Getting Started series of tutorials (here).