Stochastic weight binarization

Question

I am reading BinaryConnect paper of Bengio. I understand the whole idea and how to do deterministic binarization. However, I did not understand how the stochastic binarization works (how can we implement that) and what is the difference between stochastic binarization and deterministic binarization. Any explanation ?

Answer 1

The major difference between deterministic and stochastic binarization is the binarization method. In deterministic binarization, the result of binarization is 1 when the input value is bigger than a certain threshold (e.g., 0), and 0 otherwise. However, in stochastic binarization, the result could be 1 with probability related to the input value. You can find more details in Accelerating Neural Networks with Binary Arithmetic .