1

I'm having having some difficulty implementing a negative log likelihood function in python

My Negative log likelihood function is given as:

Screenshot of negative Log Likelihood equation

This is my implementation but i keep getting error:ValueError: shapes (31,1) and (2458,1) not aligned: 1 (dim 1) != 2458 (dim 0)

def negative_loglikelihood(X, y, theta):
    J = np.sum(-y @ X @ theta) + np.sum(np.exp(X @ theta))+ np.sum(np.log(y))
    return J

X is a dataframe of size:(2458, 31), y is a dataframe of size: (2458, 1) theta is dataframe of size: (31,1)

i cannot fig out what am i missing. Is my implementation incorrect somehow? Any help would be much appreciated. thanks

Gaurang Swarge
  • 19
  • 1
  • 5

3 Answers3

1

You cannot use matrix multiplication here, what you want is multiplying elements with the same index together, ie element wise multiplication. The correct operator is * for this purpose.

Moreover, you must transpose theta so numpy can broadcast the dimension with size 1 to 2458 (same for y: 1 is broadcasted to 31.)

x = np.random.rand(2458, 31)
y = np.random.rand(2458, 1)
theta = np.random.rand(31, 1)

def negative_loglikelihood(x, y, theta):
    J = np.sum(-y * x * theta.T) + np.sum(np.exp(x * theta.T))+ np.sum(np.log(y))
    return J
negative_loglikelihood(x, y, theta)

>>> 88707.699

EDIT: your formula includes a y! inside the logarithm, you should also update your code to match.

Thibault D.
  • 1,567
  • 10
  • 23
0

If you look at your equation you are passing yixiθ is Summing over i=1 to M so it means you should pass the same i over y and x otherwise pass the separate function over it.

Maddy6
  • 367
  • 1
  • 3
  • 14
0
Negative log-likelihood loss formula: 
            n   | log10(ai)        if yi = 1
L(a,y) = -Sigma | 
          n - 1 | log10(1-ai)      if yi = 0


def neg_log_lhood(a, y):
    mask = np.asarray(y, dtype=bool)
    na = np.array(a)
    na[mask] = np.log10(na[mask])
    na[~mask] = np.log10(1 - na[~mask])
    return -np.sum(na)

y = [0 ,0, 1, 1, 0]
a = [0.2, 0.4, 0.6, 0.8, 0.1]

neg_log_lhood(a, y)

ret: 0.6832750158095007

legale
  • 612
  • 7
  • 9