2

I would like to quantize embeddings to a single signed byte in each dimension. If I try to do this by scaling the values to [-127, 128], then casting to tf.int8, recasting to tf.float32 and rescaling to the original [-1, 1] range, I get the following error:

ValueError: No gradients provided for any variable

The same training script works fine without the quantization step.

According to this thread, quantized ops will be a future feature of tensorflow. In the mean time is there a good work-around for this simple quantization scenario?

RobR
  • 2,160
  • 2
  • 19
  • 32
  • 1
    why not skip the tf.int8 cast stage while training? Mathematically, when you introduce quantization between your cost and your variables, the gradients become zero – Yaroslav Bulatov Apr 13 '16 at 00:23
  • Thanks. Makes sense. But is it obvious that the local minima for a continuous embedding is the same or very similar to the best choice for a highly quantized embedding? – RobR Apr 13 '16 at 02:05
  • It seems to be good enough -- that's how quantized models are typically trained. You could probably get better accuracy by customizing your training procedure to be aware of quantization but it would need some tricks – Yaroslav Bulatov Apr 13 '16 at 02:42

0 Answers0