Identify Required Placeholders in GCMLE Model

Question

I have deployed a model a few weeks ago and I have lost track of which input features are required for the model (yes I know I should have kept better track of this). When I run the following command

gcloud ml-engine predict \
--model $MODEL_NAME \
--version v1 \
--json-instances \
../test.json

I get the following err:

{
  "error": "Prediction failed: Exception during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details=\"input size does not match signature\")"
}

I know that my issue is because there is some "required" placeholder that I have not included in my json request (test.json) but I cannot figure out any way to retroactively find out which placeholder is missing.

The issue is the same as this question where the poster was able to submit the prediction when he included the missing "threshold" tensor.

What would be the easiest way to look at the expected inputs for a given model?

Answer 1

The service itself does not yet allow you to query the signature of your model, so my recommendation is to use saved_model_cli where possible (assuming you didn't delete the original model). Something like:

gcloud ml-engine versions describe v1 --model census | grep deploymentUri

Which will output something like:

deploymentUri: gs://my_bucket/path/to/Servo/1488268526779

Now, use saved_model_cli:

saved_model_cli show --all --dir gs://my_bucket/path/to/Servo/1488268526779

Which will output something like:

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['age'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1)
    name: Placeholder_8:0
inputs['capital_gain'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1)
    name: Placeholder_10:0
inputs['capital_loss'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1)
    name: Placeholder_11:0
inputs['education'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_2:0
inputs['education_num'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1)
    name: Placeholder_9:0
inputs['gender'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder:0
inputs['hours_per_week'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1)
    name: Placeholder_12:0
inputs['marital_status'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_3:0
inputs['native_country'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_7:0
inputs['occupation'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_6:0
inputs['race'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_1:0
inputs['relationship'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_4:0
inputs['workclass'] tensor_info:
    dtype: DT_STRING
    shape: (-1)
    name: Placeholder_5:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
    dtype: DT_INT64
    shape: (-1)
    name: predictions/classes:0
outputs['logistic'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: predictions/logistic:0
outputs['logits'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: add:0
outputs['probabilities'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 2)
    name: predictions/probabilities:0
Method name is: tensorflow/serving/predict

Answer 2

Assuming you have the tensorflow graph associated with your Saved Model. If you can print the graph (print the protobuffer in text format) or visualize with a tool, you can see the "signature_def" field within your graph. That should tell you the inputs and the outputs for your graph. You can compare what tensors you are sending in your request against the inputs that the graph expects.