3

What is the best way to use CloudML prediction API in production service?

I've seen: https://cloud.google.com/ml/docs/quickstarts/prediction but it relies on gcloud tool

I'm looking into solution that doesn't depend on having gcloud installed and initialized on machine making the request. It would be great to have solution that works on GCP, AWS and possibly other clouds.

Thanks

Michal K
  • 205
  • 1
  • 5

2 Answers2

8

I'll show you how to authenticate your production environment to use CloudML online prediction. The CloudML quickstarts use gcloud to authenticate the end-user via username, password etc. gcloud doesn’t scale well to environments with 100’s of machines starting and stopping. Below, I walk you through the steps of creating a Cloud service account and generating a private key to which will allow your production instances to identify themselves to the Google servers. See authentication documentation here.

Here's a recipe you can use.

PROJECT=
MODEL_NAME=
SERVICE_ACCOUNT_PREFIX=cloud-ml-predict
SERVICE_ACCOUNT="${SERVICE_ACCOUNT_PREFIX}@${PROJECT}.iam.gserviceaccount.com"

These steps have to be done only once, and will create a service account and private key for you.

# Make a new service account
gcloud iam service-accounts create  ${SERVICE_ACCOUNT_PREFIX} \
  --display-name ${SERVICE_ACCOUNT_PREFIX}

# Provide correct role to service account permissions:
gcloud projects add-iam-policy-binding $PROJECT \
  --member "serviceAccount:$SERVICE_ACCOUNT" --role roles/viewer

# Create private key for the service account:
gcloud iam service-accounts keys create --iam-account \
  $SERVICE_ACCOUNT private_key.json

Now that we have a private key (in private_key.json) we can call the prediction API from any machine that has the googleapiclient Python library. Now from any machine with or without gcloud you only need to include the following lines to access the CloudML prediction service via HTTP

scopes = ['https://www.googleapis.com/auth/cloud-platform']
credentials = ServiceAccountCredentials.from_json_keyfile_name(key_filename, scopes=scopes)
ml_service = discovery.build('ml', 'v1beta1', credentials=credentials)

Finally, here's a worked example assuming you have a the MNIST model deployed from the quickstarts.

cat > key_pair_cloud_ml_serve.py <<EOD
from googleapiclient import discovery
import json
from oauth2client.service_account import ServiceAccountCredentials
import sys

def get_mnist_prediction(ml_service, project, model_name, instance):
  parent = 'projects/{}/models/{}'.format(project, model_name)
  request_dict = {'instances': [json.loads(instance)]}

  request = ml_service.projects().predict(name=parent, body=request_dict)
  print request.execute()  # waits till request is returned

if __name__ == '__main__':
  usage_str = 'usage: python prog private_key.json MODEL_NAME data/predict*json'
  assert len(sys.argv) == 4, usage_str

  key_file = sys.argv[1]
  model_name = sys.argv[2]
  data_file = sys.argv[3]

  scopes = ['https://www.googleapis.com/auth/cloud-platform']
  credentials = ServiceAccountCredentials.from_json_keyfile_name(key_file,
scopes=scopes)
  ml_service = discovery.build('ml', 'v1beta1', credentials=credentials)
  with open(key_file) as ff:
    project = json.load(ff)['project_id']


  with open(data_file) as ff:
    for ii, instance in enumerate(ff):
      get_mnist_prediction(ml_service, project, model_name, instance)
EOD

And from within the mnist/deployable folder of the Cloud ML samples we call our code...

python key_pair_cloud_ml_serve.py private_key.json \
  $MODEL_NAME data/predict_sample.tensor.json


{u'predictions': [{u'prediction': 5, u'key': 0, u'scores': [0.04025577753782272, 0.00042669562390074134, 0.005919951014220715, 0.4221051335334778, 2.2986243493505754e-05, 0.5084351897239685, 0.0007824163185432553, 0.01125132292509079, 0.008616944774985313, 0.0021835025399923325]}]}

Voila! We used a private key and never needed to use gcloud for authentication or querying our prediction model!

JoshGC
  • 469
  • 2
  • 7
  • How do you choose which project the service account will be associated with in the step `gcloud iam service-accounts create ${SERVICE_ACCOUNT_PREFIX} \ --display-name ${SERVICE_ACCOUNT_PREFIX}` – Andrew Jun 01 '17 at 20:43
  • Andrew, I don't think it matters. The project that owns the model will still be billed and still have to allow the service account access (regardless of which project owns it). The project that owns the service account merely retains the ability to delete the service account. – JoshGC Jun 02 '17 at 14:42
4

You can use the JSON API.

For online predictions: https://cloud.google.com/ml/reference/rest/v1beta1/projects/predict

For batch predictions: https://cloud.google.com/ml/reference/rest/v1beta1/projects.jobs/create

Alexey Surkov
  • 396
  • 2
  • 6