I'm working on predicting a regression model on Lasso.
- I have a total of 137 train data, and 100000 test data to predict the total revenue.
- to build the model I split train data into train and test ( train = 96, test = 97-137).
- When I ran the lasso regression to predict the testing data I received the following error: Error in cbind2(1, newx) %*% nbeta : Cholmod error 'X and/or Y have wrong dimensions' at file ../MatrixOps/cholmod_sdmult.c, line 88
My data split is showing here:
#model 4: LASSO
y<-log(training$revenue)
X<- model.matrix(Id~Open.Date*P26*P28+sqrt(Open.Date)*P26*P28+log(Open.Date)*P26*P28 + P1+P2+P3+P4+P5+P6+P7
+P8+P9+P10+P11+P12+P13+P14+P15+P16+P17+P18+P19+P20+
P21+P22+P23+P24+P25+P27+P29+P30+P31+P32+P33+P34+P35+P36+P37, new_Data)[,-1]
X<-cbind(new_Data$ID,X) #bind a new column ID to the data set
# split X into testing, trainig/holdout and prediction as before
X.training<-X[1:96,]
X.testing<-X[97:137,]
X.prediction<-X[138:100137,]
X.training#selecting the best penalty lambda (try different values of lambda and get the value of error in Y)
crossval <- cv.glmnet(x = X.training, y = y, alpha = 1) #create cross-validation data
plot(crossval)
penalty.lasso <- crossval$lambda.min #enter code heredetermine optimal penalty parameter, lambda = -4.278
log(penalty.lasso) #see where it was on the graph, and calculates the penalty
plot(crossval,xlim=c(-6,-2),ylim=c(0,0.4)) # zoom-in
lasso.opt.fit <-glmnet(x = training, y = y, alpha = 1, lambda = penalty.lasso) #estimate the model with the optimal penalty
coef(lasso.opt.fit) #resultant model coefficientspredicting the performance ( here where I got the error)
# predicting the performance on the testing set
lasso.testing <- exp(predict(lasso.opt.fit, s = penalty.lasso, newx= X.testing))
mean(abs(lasso.testing-X.testing$revenue)/X.testing$revenue*100) #calculate and display MAPEDo you know what could be this error for and what possible ways to fix it?
Thank you
