4

My question is on how to correctly interpret (and use) the 'weights' input variable in the nls function of R for non-linear weighted least squares regression.

The solution for solving the unknown parameters in weighted least squares theory is: enter image description here

From this the variable P is the weight square matrix of size (NxN) where N is the number of data observations.

However, when I look at the nls documentation in R found here, it says the 'weights' to be input is a vector.

This has me puzzled since based on my understanding, the weights should be a square matrix. Some insights with those who have a better understanding is appreciated.

user121
  • 849
  • 3
  • 21
  • 39
  • 1
    I think the function only supports the case where P is a diagonal matrix. So the vector you supply should be equal to the number of observations and will just be the elements along the diagonal of P and all over values will be 0. – MrFlick Jul 03 '18 at 19:10
  • @MrFlick if that is the case, this is a limited case of weighted LS. Are you aware of any alternative functions for weighted nonlinear LS in R? – user121 Jul 03 '18 at 19:42
  • @user121 . if you are talking about weights of your observations for fitting a nls. then it should be a diagnal weight matrix only in my opinion. Refer this Wikipedia page on [Non-linear_least_squares](https://en.wikipedia.org/wiki/Non-linear_least_squares) – Mankind_008 Jul 03 '18 at 19:49
  • @Mankind_008 but isn't it possible that if P is a square matrix; that the off diagonals can have a value other than zero? – user121 Jul 03 '18 at 20:19
  • @Mankind_008 thanks, do you have a reference for 'Generalized Non Linear LS' which shows this? – user121 Jul 03 '18 at 21:03
  • @user121 I added summary of my comments as an answer to be helpful for others. – Mankind_008 Jul 04 '18 at 01:26

1 Answers1

3

Weight variable in regression, is a measure of how important an observation is to your model due to different reasons (eg. may be in terms of reliability of measurement or inverse of variance estimate). Therefore, some observations may be more important/ weigh higher than others.

Weight vector *P*, in matrix notation converts to a diagonal matrix enter image description here for i in {1,2,3...n,} both represents the same thing (i.e. weight of ith observation). For nls package in R you need to supply weights in vector form.

Also, it should be noted that, weighted least squares is a special variant of generalized least squares in which we use weights to counter the heteroskedasticity. If the residuals are correlated for observations, perhaps a general model might be suitable.

PS: Cross validated would be the right place to get better detailed answer. Also, It seems to be memory efficient to store a vector rather than a matrix as the number of observations grows

Mankind_008
  • 2,158
  • 2
  • 9
  • 15