2

I have been experimenting with the new 8-bit quantization feature available in TensorFlow. I could run the example given in the blog post (quantization of googlenet) without any issue and it works fine for me !!!

Now, I would like to apply the same for a simpler network. So I used a pre-trained network for CIFAR-10 (which is trained on Caffe), extracted its parameters, created corresponding graph in tensorflow, initialized the weights with this pre-trained weights and finally saved it as a GraphDef object. See this IPython Notebook for full procedure.

Now I applied the 8-bit quantization with the tensorflow script as mentioned in the Pete Warden's blog:

bazel-bin/tensorflow/contrib/quantization/tools/quantize_graph --input=cifar.pb  --output=qcifar.pb --mode=eightbit --bitdepth=8 --output_node_names="ArgMax"

Now I wanted to run the classification on this quantized network. So I loaded the new qcifar.pb to a tensorflow session and passed the image (the same way I passed it to original version). Full code can be found in this IPython Notebook.

But as you can see at the end, I am getting following error:

NotFoundError: Op type not registered 'QuantizeV2'

Can anybody suggest what am I missing here?

Abid Rahman K
  • 51,886
  • 31
  • 146
  • 157

1 Answers1

3

Because the quantized ops and kernels are in contrib, you'll need to explicitly load them in your python script. There's an example of that in the quantize_graph.py script itself:

from tensorflow.contrib.quantization import load_quantized_ops_so from tensorflow.contrib.quantization.kernels import load_quantized_kernels_so

This is something that we should update the documentation to mention!

Pete Warden
  • 2,866
  • 1
  • 13
  • 12
  • Thanks. But I am getting an importError. I am using tensorflow 0.9 using pip installation. Is there anything else to do during installation to import these modules ? – Abid Rahman K Jul 15 '16 at 05:48
  • Although I couldn't import as you do, I did a small work around as I have explained here : **http://stackoverflow.com/a/38495999/1134940**. Now everything works fine. Thank you. I would really love to see more details on documentation, also I want to try out other quantization methods. – Abid Rahman K Jul 21 '16 at 06:09
  • Which files should I include to make it work using the C interface from iOS? – Senad Oct 26 '16 at 13:46