Questions tagged [log-likelihood]

Only questions related to the implementation and usage of the mathematical function - log-Likelihood should use this tag.

Given a sample and a parametric family of distributions (i.e., a set of distributions indexed by a parameter) that could have generated the sample, the Likelihood is a function that associates to each parameter the probability (or probability density) of observing the given sample.

The log-Likelihood a function which is the natural logarithm of the Likelihood function

For many applications, the log-Likelihood, is more convenient to work with as compared to the Likelihood. This is because we are generally interested in where the Likelihood reaches its maximum value. Since the logarithm is a strictly increasing function, the logarithm of a function achieves its maximum value at the same points as the function itself, hence the log-likelihood can be used in place of the likelihood for maximum likelihood estimation and related techniques.

Finding the maximum of a function often involves taking the derivative of a function and solving for the parameter being maximized, and this is often easier when the function being maximized is a log-likelihood rather than the original likelihood function, because the probability of the conjunction of several independent variables is the product of probabilities of the variables and solving an additive equation is usually easier than a multiplicative one.

232 questions
2
votes
0 answers

Explicit likelihood of WordPiece used for pre-processing of BERT

At each iteration the WordPiece algorithm for subword tokenization merges the two symbols which increase the likelihood the most. Now, in the literature it is only mentioned that this likelihood is the likelihood of the language model (e.g., the…
2
votes
1 answer

Loss function negative log likelihood giving loss despite perfect accuracy

I am debugging a sequence-to-sequence model and purposely tried to perfectly overfit a small dataset of ~200 samples (sentence pairs of length between 5-50). I am using negative log-likelihood loss in pytorch. I get low loss (~1e^-5), but the…
2
votes
0 answers

Estimate Linear models in R with either fixed slope or intercept

I would like to estimate models with either the intercept (model a) or the slope (model b) are fixed, and also estimate unconstrained model (c), where each of these parameters was allowed to vary, and a null model (d), where none could. Then, I…
CaroleGE
  • 47
  • 5
2
votes
0 answers

Maximum Likelihood Estimates with fixed parameters using the GenericLikelihoodModel from StatsModels

I have a model with three parameters, namely: beta > 0, gamma > 0 and phi > 0. Given the data T, I am able to evaluate the MLEs for the 3 parameters. However, it is very important that I also verify the cases where (i) gamma = 1; (ii) beta = 1 and…
2
votes
1 answer

How to increase the number of values returned by R MASS::profile?

The short question is: profile() returns 12 parameter values. How can it be made to return a greater number? The motivation for my question is to reproduce Fig. 1.3 in Applied Logistic Regression 3rd Edition by David W. Hosmer Jr., Stanley Lemeshow…
Richard Careaga
  • 628
  • 1
  • 5
  • 11
2
votes
1 answer

Coding a log-likelihood function in MATLAB

I am following a very detailed post from Quantitative Finance, however, my problem is a coding one. I am trying to estimate a GARCH(1,1) model (not using a statistical toolbox and rather a long hand method, the reason for this is I really want to…
2
votes
0 answers

Maximium of an Integral

I have an Integral in terms of parameters a,b,p,q. My probability densitiy function of data x is GB2PdfCalc=function(x){ out=(abs(a)*x^(a*p-1))/(b^(a*p)*beta(p,q)*(1+(x/b)^(a))^(p+q)) return(out) } I want to an integral of this using…
luuul
  • 21
  • 2
2
votes
0 answers

Calculating Log-likelihood for the Skew Normal Distribution in R

library(sn) library(fGarch) library(maxLik) set.seed(12) nl = 100 locl = 0 scalel = 1 shapel = 1 #data y = c(rsn(n=nl, xi=locl, omega=scalel, alpha=shapel, tau=0, dp=NULL)) Assume only the shape parameter is unknown. snormFit <- function(x, ...){…
score324
  • 687
  • 10
  • 18
2
votes
2 answers

Posterior from joint normal prior distribution

I have some basic questions about gaussian inference. I have following data: (Log) dose, Number of animals, Number of deaths -0.86, 5, 0 -0.30, 5, 1 -0.05, 5, 3 0.73, 5, 5 EDIT: I'm assuming a simple regression model for the dose response logit(θ)…
2
votes
1 answer

R: loglikelihood of Saturated Model in GLM

Let LL = loglikelihood Residual Deviance = 2(LL(Saturated Model) - LL(Proposed Model)) However, when I use glm function, it seems that Residual Deviance = -2LL(Proposed Model) For example, mydata <-…
ming gao
  • 53
  • 8
2
votes
1 answer

Sample from a custom likelihood function

I have the following likelihood function which I used in a rather complex model (in practice on a log scale): library(plyr) dcustom=function(x,sd,L,R){ R. = (log(R) - log(x))/sd L. = (log(L) - log(x))/sd ll = pnorm(R.) - pnorm(L.) …
Wave
  • 1,216
  • 1
  • 9
  • 22
2
votes
1 answer

Maximizing Likelihood, Julia

I have a log-likelihood function and I want to maximize it in respect to theta (N), and it is defined as: function loglik(theta,n,r) N=theta;k=length(n); …
Alex
  • 99
  • 1
  • 7
1
vote
2 answers

Calculating an integral of a Bernoulli likelihood function in R

I am trying to integrate a very simple likelihood function (three Bernoulli trials) in R, but it seems that I have not understood how integrate function works. My code is as follows: integrate( function(theta) prod( dbinom( x…
1
vote
0 answers

Calualte the model coefficients using maximum likelihood estimator for non-linear curve

I have a non-linear dataset that follows a certain distribution as such: import numpy as np from matplotlib import pyplot as plt ## Define the curve def curve(t,α,ξ,β1,β2): ## Model parameters: (α, ξ, β1, β2) ## Covariate-1 of…
1
vote
0 answers

How to implement MLE in python not via arch library?

I tried to use the MLE method to estimate the parameters of an actuarial model (the Vasicek model) which is similar to the GARCH model (similar in the fact that in both models there are 3 parameters whose value needs to be guessed using MLE). I know…
1 2
3
15 16