Tensorflow: Add small number before division for numerical stability

Question

In order to prevent divisions by zero in TensorFlow, I want to add a tiny number to my dividend. A quick search did not yield any results. In particular, I am interested in using the scientific notation, e.g.

a = b/(c+1e-05)

How can this be achieved?

Assuming a,b and c are tensors. The formula you have written will work as expected. 1e-5 will be broadcasted and added on the tensor c. — Souradeep Nanda, Mar 23 '19 at 03:36
Tensorflow automatically typecasts `1e-5` to `tf.constant(1e-5)`. You can write it manually if you want. — Souradeep Nanda, Mar 23 '19 at 05:05
Thank you. Do you mind posting this as an answer to the question below? — whiletrue, Mar 23 '19 at 10:01

score 1 · Accepted Answer · answered Mar 24 '19 at 03:54

Assuming a, b and c are tensors. The formula you have written will work as expected. 1e-5 will be broadcasted and added on the tensor c. Tensorflow automatically typecasts the 1e-5 to tf.constant(1e-5).

Tensorflow however has some limitations with non-scalar broadcasts. Take a look at my other answer.

Tensorflow: Add small number before division for numerical stability

1 Answers1