Stack Overflow
Stack Overflow

Questions tagged [hessian-matrix]

Hessian matrix is an N x N matrix containing second partial derivatives of a function with respect to its input variables.

Hessian matrices are used extensively in second-order optimization methods such as Newton's method. They give information about the curvature around a point x0 on a function surface.

119 questions
4
votes
1 answer

Faster way to calculate the Hessian / Fisher Information Matrix of a nnet::multinom multinomial regression in R using Rcpp & Kronecker products

It appears that for larger nnet::multinom multinomial regression models (with a few thousand coefficients), calculating the Hessian (the matrix of second derivatives of the negative log likelihood, also known as the observed Fisher information…
Tom Wenseleers
  • 7,535
  • 7
  • 63
  • 103
4
votes
1 answer

How to implement a negative binomial loss function in python to use in light GBM?

I have a machine learning problem that I believe the negative binomial loss function would fit well, but the light gbm package doesn't have it as a standard, I'm trying to implement it, but I'm don't know how to get Gradient and Hessian, does anyone…
Adauto.Almeida
  • 113
  • 6
4
votes
0 answers

How to compute the second derivatives (diagonal of the Hessian) in TensorFlow 2.0

I would like to compute the second derivatives (diagonal hessian) for all the components of all my variables in TensorFlow 2.0. I would like to autograph such a function as well. I have it working in eager-mode on Google Colab (with a little test…
Nicholas Vadivelu
  • 61
  • 2
4
votes
1 answer

Use tf.gradients or tf.hessians on flattened parameter tensor

Let's say I want to compute the Hessian of a scalar-valued function with respect to some parameters W (e.g the weights and biases of a feed-forward neural network). If you consider the following code, implementing a two-dimensional linear model…
lfaury
  • 98
  • 7
4
votes
0 answers

Statistical difference between two curves

I have prey consumption data on two species and I am trying to determine the degree to which their functional response curves differ. Here is a small example of my data (one separate Excel file for each species, the non-native had 6 repetitions for…
user6480584
  • 55
  • 1
  • 7
4
votes
1 answer

Retrieve approximate Hessian inverse from L-BFGS-B

With the L-BFGS-B minimizer in scipy, is it possible to retrieve the approximate inverse Hessian that's calculated internally? Having it in the implicit factored form, so that it's possible to compute arbitrary inverse Hessian matrix - vector…
Robert T. McGibbon
  • 5,075
  • 3
  • 37
  • 45
3
votes
1 answer

Creating a Custom Objective Function in for XGBoost.XGBRegressor

So I am relatively new to the ML/AI game in python, and I'm currently working on a problem surrounding the implementation of a custom objective function for XGBoost. My differential equation knowledge is pretty rusty so I've created a custom obj…
jpb
  • 31
  • 1
  • 2
3
votes
1 answer

Compute hessian with respect to several variables in tensorflow

Computing Hessian in tensorflow is quite easy: x = tf.Variable([1., 1., 1.], dtype=tf.float32, name="x") f = (x[0] + x[1] ** 2 + x[0] * x[1] + x[2]) ** 2 hessian = tf.hessians(f, x) This correctly returns [[ 8., 20., 4.], [20., 34., 6.], […
Ruggero Turra
  • 16,929
  • 16
  • 85
  • 141
3
votes
1 answer

XGBoost python custom objective function

According to documentation, http://xgboost.readthedocs.io/en/latest/python/python_api.html if we want to define custom objective function, it should have signature objective(y_true, y_pred) -> grad, hess where hess: array_like of shape…
Evgeniy1089
  • 31
  • 1
  • 2
3
votes
1 answer

Retrieve optimization results from MLE by scipy.stats.fit()?

I am trying to estimate different distributions' parameters via scipy.stats."some_dist".fit(), and I'm having extreme difficulty retrieving any information on the optimization process being used. Specifically, I am looking for the Hessian, which…
Coolio2654
  • 1,589
  • 3
  • 21
  • 46
3
votes
0 answers

Hessian matrix with optim or numderiv package?

I do maximum likelihood estimation for a loglikelihood function of a poisson distribution. After the estimation i want to compute the standard errors of the coeffients. For that i need the hessian matrix. Now i dont know which function i should use…
Dima Ku
  • 243
  • 2
  • 11
3
votes
0 answers

Tensorflow, how to compute broadcast product of 2 gradients?

I'm trying to compute a matrix H as follows: Where L is a tensor of shape (?,N) and z is a variable vector shape (M). Each element of H is a broadcast product of 2 gradients of L which respect to two elements of vector z. The tf.gradients(L,z[i]) *…
Son Nguyen
  • 33
  • 4
3
votes
1 answer

The curious case of ARIMA modelling using R

I observed something strange while fitting an ARMA model using the function arma{tseries} and arima{stats} in R. There is a radical difference in the estimation procedures adopted by the two functions, Kalman filter in arima{stats} as opposed to ML…
Shreyes
  • 3,601
  • 1
  • 17
  • 16
3
votes
4 answers

R Hessian Matrix

I need to create the Hessian matrix of a function given as: func <- expression(sin(x+y)+cos(x-y)) vars <- c("x", "y") I need the second order derivatives as expressions too, and I need to evaluate them lot of times, so I made a list of first order…
David Angyal
  • 231
  • 4
  • 13
3
votes
2 answers

Matlab fminunc calculate standard errors MLE

I have a problem when trying to calculate standard errors of estimates from fminunc. My estimation technique is Maximum likelihood Estimation. I've tried two ways as below, both failed: The Hessian matrix from fminunc results is non-invertible, so…
user2595696
  • 31
  • 1
  • 2
1
2
3 4 5 6 7 8