How to feed Cifar10 trained model with my own image and get label as output?

Question

I am trying to use the trained model based on the Cifar10 tutorial and would like to feed it with an external image 32x32 (jpg or png).
My goal is to be able to get the label as an output. In other words, I want to feed the Network with a single jpeg image of size 32 x 32, 3 channels with no label as an input and have the inference process give me the tf.argmax(logits, 1).
Basically I would like to be able to use the trained cifar10 model on an external image and see what class it will spit out.

I have been trying to do that based on the Cifar10 Tutorial and unfortunately always have issues. especially with the Session concept and the batch concept.

Any help doing that with Cifar10 would be greatly appreciated.

Here is the implemented code so far with compilation issues :

#!/usr/bin/env python

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from datetime import datetime
import math
import time

import tensorflow.python.platform
from tensorflow.python.platform import gfile
import numpy as np
import tensorflow as tf

import cifar10
import cifar10_input
import os
import faultnet_flags
from PIL import Image

FLAGS = tf.app.flags.FLAGS

def evaluate():

  filename_queue = tf.train.string_input_producer(['/home/tensor/.../inputImage.jpg'])

  reader = tf.WholeFileReader()
  key, value = reader.read(filename_queue)

  input_img = tf.image.decode_jpeg(value)

  init_op = tf.initialize_all_variables()

# Problem in here with Graph / session
  with tf.Session() as sess:
    sess.run(init_op)

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)

    for i in range(1): 
      image = input_img.eval()

    print(image.shape)
    Image.fromarray(np.asarray(image)).show()

# Problem in here is that I have only one image as input and have no label and would like to have
# it compatible with the Cifar10 network
    reshaped_image = tf.cast(image, tf.float32)
    height = FLAGS.resized_image_size
    width = FLAGS.resized_image_size
    resized_image = tf.image.resize_image_with_crop_or_pad(reshaped_image, width, height)
    float_image = tf.image.per_image_whitening(resized_image)  # reshaped_image
    num_preprocess_threads = 1
    images = tf.train.batch(
      [float_image],
      batch_size=128,
      num_threads=num_preprocess_threads,
      capacity=128)
    coord.request_stop()
    coord.join(threads)

    logits = faultnet.inference(images)

    # Calculate predictions.
    #top_k_predict_op = tf.argmax(logits, 1)

    # print('Current image is: ')
    # print(top_k_predict_op[0])

    # this does not work since there is a problem with the session
    # and the Graph conflicting
    my_classification = sess.run(tf.argmax(logits, 1))

    print ('Predicted ', my_classification[0], " for your input image.")


def main(argv=None):
  evaluate()

if __name__ == '__main__':
  tf.app.run() '''

Olivier Moindrot · Accepted Answer · 2016-06-02T08:02:51.143

4

Some basics first:

First you define your graph: image queue, image preprocessing, inference of the convnet, top-k accuracy
Then you create a tf.Session() and work inside it: starting the queue runners, and calls to sess.run()

Here is what your code should look like

# 1. GRAPH CREATION 
filename_queue = tf.train.string_input_producer(['/home/tensor/.../inputImage.jpg'])
...  # NO CREATION of a tf.Session here
float_image = ...
images = tf.expand_dims(float_image, 0)  # create a fake batch of images (batch_size=1)
logits = faultnet.inference(images)
_, top_k_pred = tf.nn.top_k(logits, k=5)

# 2. TENSORFLOW SESSION
with tf.Session() as sess:
    sess.run(init_op)

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)

    top_indices = sess.run([top_k_pred])
    print ("Predicted ", top_indices[0], " for your input image.")

EDIT:

As @mrry suggests, if you only need to work on a single image, you can remove the queue runners:

# 1. GRAPH CREATION
input_img = tf.image.decode_jpeg(tf.read_file("/home/.../your_image.jpg"), channels=3)
reshaped_image = tf.image.resize_image_with_crop_or_pad(tf.cast(input_img, width, height), tf.float32)
float_image = tf.image.per_image_withening(reshaped_image)
images = tf.expand_dims(float_image, 0)  # create a fake batch of images (batch_size = 1)
logits = faultnet.inference(images)
_, top_k_pred = tf.nn.top_k(logits, k=5)

# 2. TENSORFLOW SESSION
with tf.Session() as sess:
  sess.run(init_op)

  top_indices = sess.run([top_k_pred])
  print ("Predicted ", top_indices[0], " for your input image.")

edited Jun 02 '16 at 08:02

answered Jun 01 '16 at 22:31

Olivier Moindrot

27,908
11
92
91

One minor suggestion: if the graph is truly intended to run over a *single* image, I'd recommend replacing the `tf.train.string_input_producer()` and reader pipeline with `image = tf.decode_jpeg(tf.read_file("/home/tensor/.../inputImage.jpg"))`, and then you don't need the queue runners. – mrry Jun 01 '16 at 23:26
You are right, I updated my answer to make it more complete – Olivier Moindrot Jun 02 '16 at 08:05
Thank you to all this seems to have worked - what I do not understand however is why aren't we using the information stored in the Saver ? ```with tf.Session() as sess: ckpt = tf.train.get_checkpoint_state(FLAGS.checkpoint_dir) if ckpt and ckpt.model_checkpoint_path: saver.restore(sess, ckpt.model_checkpoint_path) else: print('No checkpoint file found') return ``` – nsalman Jun 06 '16 at 07:12
You can of course use a previously saved model, I did not include it in my answer for clarity. – Olivier Moindrot Jun 06 '16 at 08:47
1

When restoring the previously trained cifar10 model AND using a single image configuration like above, an error occurs as the expected shape of tensors in the model are based on the batch size when training. Is it possible to use placeholders with the training input pipeline to fix this? http://stackoverflow.com/questions/38177753/tensorflow-inference-with-single-image-with-cifar-10-example – DMTishler Oct 27 '16 at 00:20
@DMTishler : So you can use placeholders with `None` batch size, so that they can work with any size of batch_size: `tf.placeholder(tf.float32, [None, 32, 32, 3])`. You can also create a fake batch of images based on the single image repeated `batch_size` times and give that to the network. – Olivier Moindrot Oct 27 '16 at 00:26
Yeah the fake batch method was how I originally got around the issue. But RE the use of placeholders, how is that implemented along side the queue runners in the cifar10 example? I thought they used that method instead of a feed dict so I am unsure where to tie in a placeholder for the input. – DMTishler Nov 10 '16 at 19:55
@DMTishler : I'm not sure I understand your question, maybe you can post it in a separate post. One thing to know is that you can feed values for any node in the graph, even if they are not placeholders. – Olivier Moindrot Nov 10 '16 at 19:59
It seems like `tf.image.per_image_withening` is replaced by `tf.image.per_image_standardization` due to https://github.com/tensorflow/models/issues/693 – So S Feb 27 '17 at 21:59

score 0 · Answer 2 · edited May 23 '17 at 12:00

The original source code in cifar10_eval.py can also be used for testing own individual images as it is shown in the following console output

nbatfai@robopsy:~/Robopsychology/repos/gpu/tensorflow/tensorflow/models/image/cifar10$ python cifar10_eval.py --run_once True 2>/dev/null
[ -0.63916457  -3.31066918   2.32452989   1.51062226  15.55279636
-0.91585422   1.26451302  -4.11891603  -7.62230825  -4.29096413]
deer
nbatfai@robopsy:~/Robopsychology/repos/gpu/tensorflow/tensorflow/models/image/cifar10$ python cifar2bin.py matchbox.png input.bin 
nbatfai@robopsy:~/Robopsychology/repos/gpu/tensorflow/tensorflow/models/image/cifar10$ python cifar10_eval.py --run_once True 2>/dev/null
[ -1.30562115  12.61497402  -1.34208572  -1.3238833   -6.13368177
-1.17441642  -1.38651907  -4.3274951    2.05489922   2.54187846]
automobile
nbatfai@robopsy:~/Robopsychology/repos/gpu/tensorflow/tensorflow/models/image/cifar10$

and code snippet

#while step < num_iter and not coord.should_stop():
# predictions = sess.run([top_k_op])
print(sess.run(logits[0]))
classification = sess.run(tf.argmalogits[0], 0))
cifar10classes = ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"]
print(cifar10classes[classification])

#true_count += np.sum(predictions)
step += 1

# Compute precision @ 1.
precision = true_count / total_sample_count
# print('%s: precision @ 1 = %.3f' % (datetime.now(), precision))

More details can be found in the post How can I test own image to Cifar-10 tutorial on Tensorflow?

How to feed Cifar10 trained model with my own image and get label as output?

2 Answers2

EDIT:

Linked