How to change prediction threshold using Google AutoML?

Question

After creating Model in google AutoML we can use the provided python code to make a prediction. Here's the code :

import sys

from google.cloud import automl_v1beta1
from google.cloud.automl_v1beta1.proto import service_pb2


def get_prediction(content, project_id, model_id):
  prediction_client = automl_v1beta1.PredictionServiceClient()

  name = 'projects/{}/locations/us-central1/models/{}'.format(project_id, model_id)
  payload = {'image': {'image_bytes': content }}
  params = {}
  request = prediction_client.predict(name, payload, params)
  return request  # waits till request is returned

if __name__ == '__main__':
  file_path = sys.argv[1]
  project_id = sys.argv[2]
  model_id = sys.argv[3]

  with open(file_path, 'rb') as ff:
    content = ff.read()

  print get_prediction(content, project_id,  model_id)

I realize that it will only print detection result that has score above threshold value = 0.5 . Example output :

payload {
  classification {
    score: 0.562688529491
  }
  display_name: "dog"
}

How to print the other detection results that has score below threshold 0.5 (e.g. change threshold to 0.3) ?

score 4 · Accepted Answer · edited Apr 12 '19 at 09:12

4

See the api documentation here

params

Object with string properties

Additional domain-specific parameters, any string must be up to 25000 characters long.

For Image Classification:

score_threshold - (float) A value from 0.0 to 1.0. When the model makes predictions for an image, it will only produce results that have at least this confidence score threshold. The default is 0.5.

The actual description of the field in the proto is

map<string,string> params;

So you would change your params variable that you have set to an empty dict. Change the params variable to : params = {"score_threshold": "0.3"} will work.

edited Apr 12 '19 at 09:12

gameon67

3,981
5
35
61

answered Apr 12 '19 at 06:26

West

722
7
16

In the params variable. – West Apr 12 '19 at 07:21
can you give an example? something like `prediction_client.score_threshold(0.3)` doesn't work – gameon67 Apr 12 '19 at 07:29
The best way is to review the protobuf guide for python there are examples there. – West Apr 12 '19 at 07:31
Ok, I don't really understand, let's say I change the `params` to dict as you said, then what to do? How do I pass the `params` variable to prediction function `request = prediction_client.predict(name, payload, params)` ? Where should I define the `score_threshold()`? – gameon67 Apr 12 '19 at 08:21
```params = {"score_threshold": score_threshold}``` – West Apr 12 '19 at 08:30
if I change `score_threshold` to 0.3 I got this error `TypeError: 0.3 has type , but expected one of: (, )` – gameon67 Apr 12 '19 at 08:44
And if you change it to "0.3" ? – West Apr 12 '19 at 08:48
Perfect!! Thanks for the help, I'll edit your question for the additional information – gameon67 Apr 12 '19 at 09:03

How to change prediction threshold using Google AutoML?

1 Answers1