1

This is a follow-up of another question by me : Error with 8-bit Quantization in Tensorflow

Basically, I would like to install the Tensorflow with 8-bit quantization support. Currently, I installed Tensorflow 0.9 with pip installation method on CentOS 7 machine (without GPU support).

I could compile and run the codes as given in Pete Warden's blog post. But, I can't import the functions given in Pete Warden's reply. I would like to add the quantization support. I couldn't find any details about the quantization part in the Tensorflow documentation also.

Can anybody share the details on how to do it?

Community
  • 1
  • 1
Abid Rahman K
  • 51,886
  • 31
  • 146
  • 157
  • Update the tensorflow version solves this load problem. [tensorflow0.10](http://stackoverflow.com/a/38729111/714955) – samuel Aug 02 '16 at 19:41

2 Answers2

3

For time-being, I could figure out a method to do this. But still waiting for official method from any TensorFlow developers.

  1. First install the tensorflow ( I tried both source installation as well as PIP installation, both are fine)
  2. Get the tensorflow source from the Github repo and go to the tensorflow root directory (I would call it tensorflow_root.
  3. Now compile the quantization script as given in Pete Warden's blog

bazel build tensorflow/contrib/quantization/tools:quantize_graph

This wil create ops libraries for quantized versions. Go to tensorflow_root/bazel-bin/tensorflow/contrib/quantization and you should see two library files : _quantized_ops.so and kernels/_quantized_kernels.so

  1. Now in your script, along with tensorflow, you should import these two library files also, using a dedicated tensorflow function

You can do it using tf.load_op_library() function

import tensorflow as tf
qops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/_quantized_ops.so')
qkernelops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/kernels/_quantized_kernels.so')
Abid Rahman K
  • 51,886
  • 31
  • 146
  • 157
0

Are you using GPU or CPU version of tensorflow.

for example it does not work on CPU, though as Abid has mentioned loading the ops libraries from local working directory.

But what is the point of using bazel build when latest version has these ops integrated and being imported without error.

$ khemant@saturn:~/DeepLearning/TF$ python -c "import tensorflow as tf; print(tf.__version__)"

0.12.0-rc0

$ khemant@saturn:~/DeepLearning/TF$ python

Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>> from tensorflow.contrib.quantization import load_quantized_ops_so

Traceback (most recent call last): File "", line 1, in ImportError: cannot import name load_quantized_ops_so

>>> import tensorflow as tf

>>> from tensorflow.contrib.quantization import load_quantized_ops_so

Traceback (most recent call last): File "", line 1, in ImportError: cannot import name load_quantized_ops_so `

Hemant K.
  • 41
  • 6
  • Latest tensorflow (v0.10+) doesn't need this. It is directly loaded and will work out-of-box. I was using v0.9 while posting this question. – Abid Rahman K Dec 09 '16 at 05:44
  • wha it shows above is tensroflow v0.12. Error still persists despite tensforflow being imported. why? where is the quantization api usgae details in python if import of load_quantized_ops_so is successfull, say ? – Hemant K. Dec 13 '16 at 12:44
  • what is the API description to use without bazel build. I am able to import successfully in python quantized_ops and – Hemant K. Dec 15 '16 at 04:39