0

Suppose I have two tensors, p1 and p2 in tensorflow of the same shape which contain probilities, some of which might be zero or one. Is their and elegant way of calculating the log-likelihood pointwise: p1*log(p2) + (1-p1)*log(1-p2)?

Implementing it naively using the tensorflow functions

p1*tf.log(p2) + (1-p1)*tf.log(1-p2)

risks calling 0*tf.log(0) which will give a nan.

patapouf_ai
  • 17,605
  • 13
  • 92
  • 132

2 Answers2

1

As an initial hack (there most be a better solution) I add an epsilon inside the log:

eps = 1e-10
p1*tf.log(p2+eps) + (1-p1)*tf.log(1-p2+eps)

which prevents a log(0).

patapouf_ai
  • 17,605
  • 13
  • 92
  • 132
0

Please take a look at the CRF. It contains implementation of Log Likelihood. In particular you can take a look at the implementation

user1877600
  • 627
  • 1
  • 9
  • 26
  • 1
    Thank you, however it is very difficult to see figure out what is what in this code. I would expect it to be possible to do using less then 5 lines of tensorflow code. – patapouf_ai May 09 '17 at 13:09
  • If you remove all documentary comments and input parameters validation you will get the code that is roughly 10 lines long. – user1877600 May 09 '17 at 13:16