Stack Overflow
Stack Overflow

Questions tagged [hessian-matrix]

Hessian matrix is an N x N matrix containing second partial derivatives of a function with respect to its input variables.

Hessian matrices are used extensively in second-order optimization methods such as Newton's method. They give information about the curvature around a point x0 on a function surface.

119 questions
2
votes
1 answer

Very small numerical issues with hessian symmetry and sparse command

I am using IPOPT in MATLAB to run an optimization and I am running into some issues where it says: Hessian must be an n x n sparse, symmetric and lower triangular matrix with row indices in increasing order, where n is the number of…
Huckleberry Febbo
  • 303
  • 2
  • 14
2
votes
1 answer

Bayesian Neural Network: Computation of Hessian

I'm trying to code in Python several types of ANN algorithms in order to get a better understanding/intuition of those. I'm not using Scikit-learn or any other ready-to-go packages since my goal is rather educational than practical. As an example…
Nikolay
  • 21
  • 2
2
votes
0 answers

Most efficient way to calculate hessian of cost function in neural network

I am coding a MLP network and I would like to implement the levenberg-marquardt algorithm. With levenberg-marquardt, the weights' update after each iteration is given by this formula: W(t+1) = W(t) - (H(t)+ l(t)*I)^-1 * J // W(t) is the matrix of…
Arkhan
  • 99
  • 11
2
votes
2 answers

Hessian-Free Optimization versus Gradient Descent for DNN training

How do the Hessian-Free (HF) Optimization techniques compare against the Gradient Descent techniques (for e.g. Stochastic Gradient Descent (SGD), Batch Gradient Descent, Adaptive Gradient Descent) for training Deep Neural Networks (DNN)? Under what…
Dr. Prasanna Date
  • 745
  • 1
  • 9
  • 29
2
votes
1 answer

Matlab fminsearch Hessians?

A warning that Im very new to this and out of my depth, so apologies if this is novice or unclear. Im estimating parameters using fminsearch for a number of datasets and it has been suggested that we should try to plot hessians for the fit for each…
user3519116
  • 75
  • 5
2
votes
2 answers

Fastest way to create a sparse matrix of the form A.T * diag(b) * A + C?

I'm trying to optimize a piece of code that solves a large sparse nonlinear system using an interior point method. During the update step, this involves computing the Hessian matrix H, the gradient g, then solving for d in H * d = -g to get the new…
ali_m
  • 71,714
  • 23
  • 223
  • 298
2
votes
1 answer

Which results should I trust between command “Hessian” and “numericHessian”?

I am trying to get the Hessian matrix from my own data, and I have two results - using the code Hessian from library(numDeriv) using code numericHessian from library(maxLik) The result from the Hessian is very very small relative to the result…
user3319993
2
votes
1 answer

How to calculate second order derivative at output layer in neural networks?

I am trying to implement the stochastic diagonal Levenberg-Marquardt method for Convolutional Neural Network in order to back propagate for learning weights. i am new in it, and quite confused in it, so I have few questions, i hope you may help me.…
khan
  • 531
  • 6
  • 29
2
votes
2 answers

R error: Error in Hessian in OPTIM when trying to estimate using max. likelihood

I'm an R noob which might be reflected in the not so dense code - so please bear. I'm trying to estimate coefficients for a bivariate normal distribution using max. likelihood estimation. I receive errors related to the Hessian when calling the…
imfundo
  • 43
  • 4
2
votes
0 answers

Octave Error in Providing sqp with Hessian Function

I am trying to solve the following optimization problem in octave The first contraint is that A be positive semi-definite. S is a set of data points such that if (xi,xj) is in S then xi is similar to xj and D is a set of data points such that if…
mfbutner
  • 141
  • 6
1
vote
1 answer

Python Skimage: hessian() filter. Why are filter result values <= 0 set to 1?

I am currently working with ridge detection filters such as frangi(), sato() and hessian() within the python skimage package. In my project, I am using the hessian() filter to detect river-like structures in an image. Since the those filte types…
sourdough95
  • 13
  • 3
1
vote
1 answer

PyTorch: Compute Hessian matrix of the model

Say that, for some reason, I want to fit a linear regression using PyTorch, as illustrated below. How could I compute the Hessian matrix of the model to, ultimately, compute the standard error for my parameter estimates? import torch import…
Álvaro A. Gutiérrez-Vargas
  • 1,896
  • 2
  • 8
  • 24
1
vote
1 answer

The output of functorch.hessian is not understandable

I am calculating the hessian of a ( 2,2) linear model using functorch.hessian as follows model = torch.nn.Linear(2,2).to(device) inputs = torch.rand(1,2).to(device) criterion = torch.nn.CrossEntropyLoss() target=torch.ones(len(inputs),…
MMM
  • 43
  • 6
1
vote
1 answer

Edge Response Removal in SIFT

In Lowe's paper Section 4.1 the ratio of principal curvatures using the Hessian Matrix is used to eliminate points that may belong to an edge. The paper does not specify whether the Hessian matrix is computed on the original image or the DoG. In…
Nirmala
  • 19
  • 5
1
vote
1 answer

Necessity of Hessian Matrix

Hessian Matrix helps determine the saddle points, and the local extremum of a function. Source: https://machinelearningmastery.com/a-gentle-introduction-to-hessian-matrices/ Hessian Matrix is used in Newton methods to optimize functions. Of what use…
user16673741
1 2 3
4
5 6 7 8