2

I have a trained TF model that that operates on a serialized (TFRecord) input. The image data has variable shape and is converted to a 229x229x3 shape via tf.image.resize_images(...). I would like to use the gcloud ml-engine predict platform similar to this, making sure to accept any size image as input.

I get my features tensor (which is passed to the prediction graph) from the following function:

def jpeg_serving_input_fn():
  """
  Serve single jpeg feature to the prediction graph
  :return: Image as a tensor
  """
  input_features = tf.placeholder(dtype=tf.float32, shape=[None, None, 3], 
                                  name="PREDICT_PLACEHOLDER")
  features_normalized = tf.image.resize_images(input_features, [229, 229])

  image = tf.reshape(features_normalized, [1, 229, 229, 3], name="RESHAPE_PREDICT")

  inputs = {
    'image': image
  }

The tf.reshape at the end is because my prediction graph expects a tensor of shape [batch_size, 229, 229, 3]. When I run this through the engine via

gcloud ml-engine local predict \
--model-dir=trained_model/export/ \
--json-instances=img.json

I get a PredictionError:

predict_lib_beta.PredictionError: (4, "Exception during running the graph: Cannot feed value of shape (1, 1600, 2400, 3) for Tensor u'RESHAPE_PREDICT:0', which has shape '(1, 229, 229, 3)'")

It looks to me like tf.reshape is being fed the output of tf.image.resize_images which should have the correct shape. Any thoughts on what I'm doing wrong here? Thanks in advance!

fenkerbb
  • 795
  • 2
  • 7
  • 18
  • The Tensorflow bit looks correct to me; features_normalized should contain output a (229, 229, 3) shaped tensor. Could you add some debug in your function to verify it is using the model you think it is for prediction? I only just started using Google Cloud ML a few weeks ago, so I'm interested to see what the issue here is. – SuperTetelman Mar 21 '17 at 22:28
  • Adding `tf.logging.debug(features_normalized.get_shape())` prints out what I expect: `229x229x3`. BUT, it does this while building the graph during training before saving the model. When restoring the model for prediction, the shape is not repeated. This clearly makes sense, but it is strange that when I look at the graph in `tensorboard` `RESHAPE_PREDICT` is nowhere to be found. – fenkerbb Mar 22 '17 at 13:34

1 Answers1

3

It looks like the error is caused by some code that feeds the "RESHAPE_PREDICT:0" tensor (i.e. the output of the tf.reshape() op, image) rather than the "PREDICT_PLACEHOLDER:0" tensor (i.e. the input to the tf.image.resize_images() op, input_features).

Without the whole source to your trained model, it's hard to say exactly what changes are necessary, but it might be as simple as changing the definition of inputs to:

inputs = {'image': input_features}

...so that the prediction service knows to feed values to that placeholder, rather than the fixed-shape output of tf.reshape().

mrry
  • 125,488
  • 26
  • 399
  • 400
  • Thanks, this led me to the solution, and yes the problems were in how this code was used elsewhere in my code. Essentially my `inputs` dictionary is defining a format for JSON input files for prediction (go figure :-)). The values of the dictionaries are tensors whose shape get passed to `tf.saved_model.signature_def_utils.build_signature_def(...)`. Is there a good tutorial or how-to for this module? The docs look a little sparse. In any case, big thanks @mrry – fenkerbb Mar 22 '17 at 18:27
  • I think the only docs for `tf.saved_model` are these ones: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md (plus any comments on the code itself). I believe the team is working on a tutorial for using this, but it might be worth to remind them by opening a [GitHub issue](https://github.com/tensorflow/tensorflow/issues) to request better docs. – mrry Mar 22 '17 at 23:56