"system is computationally singular" brnn package

Question

I am using brnn package to fit a regularized neural network to my data. In some cases, I get an error:

 Error in solve.default(2 * beta * H + ii(2 * alpha, npar)) : 
  system is computationally singular: reciprocal condition number = 2.29108e-20

I read all the issues related to this topic on stackoverflow, but solutions are not directly applicable to my problem. So far I know:

The problem is in brnn() function, which relys on solve() function. Solution would be to reduce tolerance (tol argument in brnn package). I did reduce it, but the problem stayed.
Multicolinearity of predictors. This is not possible, since I have only one independet variable.
Here I am stuck..

GitHub site for brnn package

A fraction of my code, which can be used to reproduce an error: 1. create data

temporal_df = structure(list(x = c(-0.553333333333333, -3.56, -2.36333333333333, 
                                   1.48666666666667, 1.15, 0.636666666666667, -0.593333333333333, 
                                   -1.52, -2.56, -0.156666666666667, -1.09666666666667, 0.96, 0.02, 
                                   1.73333333333333, 0.34, 1.25666666666667, -0.396666666666667, 
                                   -1.15, 2.95, -1.95333333333333, -0.293333333333333, 4.33333333333333, 
                                   0.35, 1.41666666666667, 3.36666666666667, -1.54333333333333, 
                                   1.1, 0.32, 2.42, 0.34, -1.82333333333333, 1.88333333333333, 2.07666666666667, 
                                   1.96, 2.25333333333333, 0.303333333333333, 2.81333333333333, 
                                   -3.14, 0.776666666666667, 4.93, -2.16666666666667, 2.41333333333333, 
                                   2.23333333333333, 1.71666666666667, 0.623333333333333, 4.85666666666667, 
                                   0.436666666666667, 2.56333333333333, 2.21666666666667, 0.0133333333333334, 
                                   3.38333333333333, 1.51666666666667), MVA = c(7.1856694, 5.598461, 
                                                                                5.872606, 6.5031284, 5.6605362, 6.002758, 6.018826, 7.3664676, 
                                                                                5.7172694, 5.9872138, 6.07253916666667, 5.87814966666667, 5.132916, 
                                                                                6.26116966666667, 5.7409835, 5.75330233333333, 5.93054783333333, 
                                                                                5.52767016666667, 5.5299795, 5.8777515, 5.501568, 5.696386, 5.74542866666667, 
                                                                                5.45688033333333, 5.14158866666667, 6.22877433333333, 6.39709566666667, 
                                                                                6.82969366666667, 6.709905, 6.06170333333333, 6.11582483333333, 
                                                                                6.20273833333333, 6.709709, 6.40844766666667, 6.15858716666667, 
                                                                                5.9047125, 6.1760875, 6.86213666666667, 6.45906283333334, 7.02090133333333, 
                                                                                6.467793, 6.47158383333333, 6.76265383333333, 6.10339883333333, 
                                                                                7.23381633333333, 6.75162833333333, 6.59454716666667, 6.50917566666667, 
                                                                                6.66505483333333, 7.58141116666667, 7.15875233333333, 7.742872
                                   )), .Names = c("x", "MVA"), row.names = c(NA, -52L), class = "data.frame")

Now fit brnn model:

#install.packages('brnn')    
library(brnn)
temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-30)

Edit after the first suggested solution: One of possible solutions, would be to to set tol = 1e-6. This actually only partly saves the problem. I still get the error in about 1/3 of repetitions. Therefore, I believe there should be something else.

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
Error in solve.default(2 * beta * H + ii(2 * alpha, npar)) : 
  system is computationally singular: reciprocal condition number = 5.01465e-19

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
Error in solve.default(2 * beta * H + ii(2 * alpha, npar)) : 
  system is computationally singular: reciprocal condition number = 7.24518e-19

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753 

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
Error in solve.default(2 * beta * H + ii(2 * alpha, npar)) : 
  system is computationally singular: reciprocal condition number = 1.04673e-17

No, there are two numeric variables. Y ~ X. Y is fixed through all iterations, while X is changing in each iteration. X is calculated in rowMeans. I edited my question and added a sample of code. — JerryTheForester, Jul 14 '17 at 07:07
P.S: is there a way to copy this question to cross-validted as well? — JerryTheForester, Jul 14 '17 at 09:23

score 1 · Accepted Answer · answered Jul 14 '17 at 14:59

1

The number 1e-30 is way smaller than the typical "essentially equal to zero number" of 1e-16. That number is chosen on the basis of it being close to the square root of 2.2e-32 which is the limit of accuracy of the POSIX standard 8-byte floating point representation of "doubles". I've never seen any of the R gurus use such a small number. See this explanation on CV.com: How do I force the L-BFGS-B to not stop early? https://stats.stackexchange.com/questions/126251/how-do-i-force-the-l-bfgs-b-to-not-stop-early-projected-gradient-is-zero

It's also unnecessary since the call to brnn (sometimes) runs without error if the default tolerance is left at 1e-6:

> temporal_model <- brnn(x ~ ., data = temporal_df, neurons = 2, tol = 1e-6)
Number of parameters (weights and biases) to estimate: 6 
Nguyen-Widrow method
Scaling factor= 1.4 
gamma= 0     alpha= 0    beta= 2.3753

When this method runs without error, which is a psuedo-random occurrence, there's really no structure in your data and the predictions are uniform and pretty close to the mean.

answered Jul 14 '17 at 14:59

IRTFM

258,963
21
364
487

Thank you @42- . However, the problem is not totally solved. If you repeat the code several times, function still returns error. (see my edited question for details). But Ok, I am far from mathematician, is there a way to prevent error? – JerryTheForester Jul 14 '17 at 18:11
1

I suspect that if you were to give it data with a bit more "trend" rather than have a basically null relationship that there would be fewer errors thrown because of computationally singular events. – IRTFM Jul 14 '17 at 20:29
True. Here is the catch: this function is only a part of a bigger function, which iterates through possible predictors and tries to find the optimal one. However, some predictors have null relationship (e.g. example in the question), but I need to get around this otherwise I get an error. Therefore, possible solution would also be to supress error… – JerryTheForester Jul 15 '17 at 06:12
1

Have you looked at `?try` and `?tryCatch`? That's not suppressing an error, so much as ignoring it. – IRTFM Jul 15 '17 at 06:15

"system is computationally singular" brnn package

1 Answers1