Stack Overflow
Stack Overflow

Questions tagged [hessian-matrix]

Hessian matrix is an N x N matrix containing second partial derivatives of a function with respect to its input variables.

Hessian matrices are used extensively in second-order optimization methods such as Newton's method. They give information about the curvature around a point x0 on a function surface.

119 questions
2
votes
1 answer

Fminunc returns indefinite Hessian matrix for a convex objective

In minimizing a convex objective function, does it mean that the Hessian matrix at minimizer should be PSD? If fminunc in Matlab returns a hessian which is not psd what does it mean? am I using a wrong objective?
user806167
  • 43
  • 1
  • 5
2
votes
2 answers

How to compile a function that computes the Hessian?

I am looking to see how a function that computes the Hessian of a log-likelihood can be compiled, so that it can be efficiently used with different sets of parameters. Here is an example. Suppose we have a function that computes the log-likelihood…
asim
  • 389
  • 2
  • 10
2
votes
1 answer

Extract the diagonal elements of the Hessian in a neural network in Jax

I have a PyTree params (in my case a nested dictionary) containing my parameters of a neural network. My goal is to compute the diagonal entries of the Hessian of a loss function with respect to the parameters and store it in a PyTree of the same…
Philipp D.
  • 61
  • 3
2
votes
2 answers

Cannot find in-place operation causing "RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation:"

I'm relatively new to PyTorch and am trying to reproduce an algorithm from an academic paper that approximates a term using the Hessian matrix. I've set up a toy problem so that I can compare the results of the full Hessian with the approximation. …
jalane
  • 23
  • 1
  • 4
2
votes
0 answers

Why do the standard errors obtained via lsqcurvefit and fminunc function differ?

I am fitting a model to experimental data in Matlab. For this model I wanted to find the parameters by minimising the sum of square residues between the experimental and model data set. The model is of the form shown below: G(x) =…
Vinotharan Annarasa
  • 21
  • 1
2
votes
1 answer

Hessian matrix, how to combine Ixx & Iyy together?

"Before extracting the lines, you need to detect potential points on them. Apply a Gaussian filter first and use the Sobel filters as derivative operators. Threshold the determinant of the Hessian and then apply non-maximum suppression in 3 × 3…
Taylor
  • 21
  • 1
  • 2
2
votes
0 answers

How to obtain second derivatives of a Loss function with respect to the parameters of a neural network using gradient tape in Tensorflow eager mode

I am creating a basic auto-encoder for the MNIST dataset using TensorFlow eager mode. I would like to observe the second-order partial derivatives of my loss function with respect to the parameters of the network as it trains. Currently, calling…
Devon Jarvis
  • 118
  • 8
2
votes
1 answer

Reshaping numpy array

What I am trying to do is take a numpy array representing 3D image data and calculate the hessian matrix for every voxel. My input is a matrix of shape (Z,X,Y) and I can easily take a slice along z and retrieve a single original image. gx, gy, gz =…
dadrake
  • 112
  • 7
2
votes
1 answer

Using TensorFlow hessians for second partial derivative test

Second partial derivative test is a simple way to tell whether a critical point is a minimum, a maximum, or a saddle. I am currently toying with the idea of implementing such test for a simple neural network in tensorflow. The following set of…
anna-earwen
  • 189
  • 13
2
votes
1 answer

Substitute values for variables in the result of derive_by_array in SymPy

I'm currently working through some exercises on multivariable function calculus and thought I would have a go at making my own function to determine gradient and hessian at a defined point for any function. I'm currently having issues when…
Josmoor98
  • 1,721
  • 10
  • 27
2
votes
1 answer

Tensorflow Error when using tf.gradients and tf.hessian: TypeError: Fetch argument None has invalid type

I just started learning tensorflow and came across the following error when using the tf.gradients and tf.hessain functions. Given below is the code and error for tf.gradients. import tensorflow as tf a = tf.placeholder(tf.float32,shape = (2,2)) b =…
noob_eggplant
  • 23
  • 4
2
votes
1 answer

Return Inverse Hessian Matrix at the end of DNN Training and Partial Derivatives wrt the Inputs

Using Keras and Tensorflow as the backend, I have built a DNN that takes stellar spectra as an input (7213 data points) and output three stellar parameters (Temperature, gravity, and metallicity). The network trains well and predicts well on my test…
Starnetter
  • 837
  • 2
  • 8
  • 21
2
votes
1 answer

Warning in Rsolnp : Solution not reliable because unable to Invert Hessian

I am trying to understand why do I get warning messages while trying to solve this using solnp? Following is the message I get - solnp--> Solution not reliable....Problem Inverting Hessian. Warning message: In p0 * vscale[(neq + 2):(nc + np + 1)]…
user6787317
  • 93
  • 7
2
votes
0 answers

Compute gradient and hessian with the python package numdifftools when parameters are bounded

I have minimized a log likelihood function using scipy.minimize and I want to compute the standard-errors associated to the parameters. In cases in which parameters are not bounded, using numdifftools (nd.Gradient and nd.Hessian) works perfectly. I…
Ariane
  • 173
  • 1
  • 2
  • 10
2
votes
0 answers

Calculate variance-covariance matrix using Hessian in polr

I'm trying to calculate the variance-covariance matrix of a polr model using the Hessian matrix output from the function. This is from the example in the polr help file. house.plr <- polr(Sat ~ Infl + Type + Cont, weights = Freq, data = housing,…
Astral Traveler
  • 31
  • 3
1 2
3
4 5 6 7 8