0

I want to approximate a constant function with a sum of nonlinear functions. I can do this with ordinary least squares, but with lasso something is going wrong, probably because the function to be approximated is constant. I give a toy example in Matlab below:

t = -1:0.01:1; %horizontal axis
x = [exp(-(t+0.5).^2); exp(-t.^2); exp(-(t-0.5).^2);]'; %I use radial basis functions in this example
y = 0.7 * ones(201,1); %Approximate a constant function by a weighted sum of radial basis functions

w = y'/x'; %ordinary least squares works fine
plot(t,w*x'); hold on; plot(t,y,'--k'); axis([-1,1,0,1]); %show results

b = lasso(x,y); %lasso does not work, this gives only zeros
w = b(:,1); %zero weights
plot(t,w*x'); hold on; plot(t,y,'--k'); axis([-1,1,0,1]); %show results

I noticed Lasso subtracts the mean from the input and output first, so this would give a zero output, hence all the zero weights resulting from lasso. Is there any way to circumvent this? Or another way to get a sparse result for the weights?

user2118903
  • 401
  • 1
  • 4
  • 8

1 Answers1

0

if I'm not wrong LASSO tries to remove those predictors (input variables) which no have information about the output.

In your example no linear combination of time dependant variables that gives (fits) always the same value (a constant output), so LASSO determines that none of your inputs (and their combinations) have information about the output; they do not correlate.

LASSO is a special case of an elastic net. The 'Alpha' parameter is 1.0 by default, the higher the 'Alpha', the fewer predictors selected. As two of your predictors: x(:,1) and x(:,3) are highly correlated (r=0.77), you try to set the 'Alpha' parameter to 0.5, for example; even with really low 'Alpha's values the output is always 0. All the predictors are obviated to achieve a constant output, i.e., to achieve always the same output you do not need any input variable.

I hope not to be wrong.

PS: ordinary least squares is calculated with the backslash ('\'). See: http://www.mathworks.es/es/help/matlab/ref/lscov.html

Best!

Josue
  • 1