1

Use-case: I am trying to load a pre-trained Keras Model as .h5 file in Google App Engine. I am running App Engine on a Python runtime 3.7 and Standard Environment.

Issue: I tried using the load_model() Keras function. Unfortunately, the load_model function does require a 'file_path' and I failed to load the Model from the Google App Engine file explorer. Further, Google Cloud Storage seems not to be an option as it is not recognized as a file path.

Questions:

(1) How can I load a pretrained model (e.g. .h5) into Google App Engine (without saving it locally first)?

(2) Maybe there is a way to load the model.h5 into Google App Engine from Google Storage that I have not thought of, e.g by using another function (other than tf.keras.models.load_model()) or in another format?

I just want to read the model in order to make predictions. Writing or training the model in not required.

Cassini
  • 33
  • 7

2 Answers2

2

I finally managed to load the Keras Model in Google App Engine -- overcoming four challenges:

Solution:

First challenge: As of today Google App Engine does only provide TF Version 2.0.0x. Hence, make sure to set in your requirements.txt file the correct version. I ended up using 2.0.0b1 for my project.

Second challenge: In order to use a pretrained model, make sure the model has been saved using this particular TensorFlow Version, which is running on Google App Engine.

Third challenge: Google App Engine does not allow you to read from disk. The only possibility to read / or store data is to use memory respectively the /tmp folder (as correctly pointed out by user bhito). I ended up connecting my Gcloud bucket and loaded the model.h5 file as a blob into the /tmp folder.

Fourth challenge: By default the instance class of Google App Engine is limited to 256mb. Due to my model size, I needed to increase the instance class accordingly.

In summary, YES tf.keras.models.load_model() does work on App Engine reading from Cloud Storage having the right TF Version and the right instance (with enough memory)

I hope this will help future folks who want to use Google App Engine to deploy there ML Models.

Cassini
  • 33
  • 7
0

You will have to download the file first before using it, Cloud Storage paths can't be used to access objects. There is a sample on how to download objects in the documentation:

from google.cloud import storage


def download_blob(bucket_name, source_blob_name, destination_file_name):
    """Downloads a blob from the bucket."""
    # bucket_name = "your-bucket-name"
    # source_blob_name = "storage-object-name"
    # destination_file_name = "local/path/to/file"

    storage_client = storage.Client()

    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(source_blob_name)
    blob.download_to_filename(destination_file_name)

    print(
        "Blob {} downloaded to {}.".format(
            source_blob_name, destination_file_name
        )
    )

And then write the file to the /tmp temporary folder which is the only one available in App Engine. But you have to take into consideration that once the instance using the file is deleted, the file will be deleted as well.

Being more specific to your question, to load a keras model, it's useful to have it as a pickle, as this tutorial shows:

def _load_model():
    global MODEL
    client = storage.Client()
    bucket = client.get_bucket(MODEL_BUCKET)
    blob = bucket.get_blob(MODEL_FILENAME)
    s = blob.download_as_string()

    MODEL = pickle.loads(s)

I also have been able to found an answer to another Stackoverflow post that covers what you're actually looking for.

bhito
  • 2,083
  • 7
  • 13
  • Thanks bhito for getting back to this -- this all makes sense. However, I still struggle to access the /tmp folder in GAE. I tried the following: storage_client = storage.Client() bucket_name = 'data_flask' model = 'model.h5' bucket = storage_client.get_bucket(bucket_name) blob = bucket.blob(model) temp_path = "/tmp/my_file.h5" blob.download_to_filename(temp_path) model = load_model(temp_path) --> loading the model somehow fails. – Cassini May 12 '20 at 15:32
  • What is the error you are getting now? That would help debugging furter – bhito May 12 '20 at 18:11
  • I think what is missing there is that you would have to open the file first by doing `with open("/tmp/test.csv") as model_file: model = load_model(model_file) ` However seeing the error you are getting will help giving you a better advice. – bhito May 13 '20 at 12:13
  • I tried it your recommended way as well. However, I only get the following error -- which is not really specific: **"Error: Server Error: The server encountered an error and could not complete your request. Please try again in 30 seconds."** The deploying process works fine though. Maybe it is the Keras `load_model()` function that does not like this tmp file path? – Cassini May 13 '20 at 20:42
  • in the logs I found: **fname = os.path.abspath(fspath(fname)) TypeError: expected str, bytes or os.PathLike object, not _io.TextIOWrapper**. – Cassini May 13 '20 at 20:46
  • Hey it seems to be related to the way the file is tried to be open, have you tried opening it as a pickle? There's this [tutorial](https://cloud.google.com/community/tutorials/appengine-serve-machine-learning-model) that may come handy for you. Also there's another [SO answer](https://stackoverflow.com/a/49886929/11641073) that solves your own question as well. – bhito May 14 '20 at 07:21
  • Ok a big step forward. -- I went through the logs once again and found that *maybe* memory is too small for loading the TF model. I tried using another machine --> **instance_class: F4_HIGHMEM**. Source: https://cloud.google.com/appengine/docs/standard Now the error is gone -- let's see what the next one will be ;) Thanks bhito for all your advice. – Cassini May 14 '20 at 10:04
  • Good to know that you've managed to overcome all the errors. F4_HIGHMEM may be a bit of overkill and raise your billing expenses, depending on how much your app is consuming. You can check the App Engine dashboard to see memory consumption and maybe switch to a lower machine tier. PS: If the answer provided was useful please consider accepting it so it has more visibility to the community. I will edit it now with all the extra information that was helpful on how to open a keras model :) – bhito May 14 '20 at 10:28