-1

I found out through many research papers that Machine Learning algorithms (and more specifically CNNs/DNNs) are actually very error-tolerant applications i.e., they can survive severe numerical errors, to the point where a very acceptable quality of results and accuracy can be obtained by using 8-bit and even sub-byte integer computational operators. Some research papers for example demonstrate that it is possible to obtain good results with 4-bit Integer MAC units, making floating-point units completely non-useful for such applications.

My question is about Floating-Point, where do you think this is really mandatory? does it find any place in any Machine Learning/Artificial Intelligence sub-domains? or is it really a game in general-purpose and scientific computing only? Any pointers to some useful applications/benchmarks/platforms that really need it and rely on it?

noureddine-as
  • 453
  • 4
  • 12

1 Answers1

0

It is a design decision to make between float/int and number of bits. A good way to make this decision would be to do a post-training quantisation of the model and then see how that changes the accuracy of the model and the performance improvement. If there is enough performance improvement within an acceptable level of drop in accuracy, quantisation can be added as a post processing step to the model generation process.

This approach is not limited to scientific computing alone or any specific fields. The inception image recognition model was quantised to have a 4 times better performance for prediction time. Having said that, there is increased adoption of quantisation in models deployed on edge devices like a Jetson or a Tensforflow-Lite application

A good starting point : Tensorflow post training quantisation

Below is a benchmark from here Benchmark of quantisation in Image Recogniton models

Deepak
  • 1,347
  • 7
  • 20