2

I applied the SVM algorithm with a Radial kernel to a regression problem using the following packages: caret (train function with SVMRadial method), e1071 (svm function) and kernlab (ksvm function). For functions on caret and kernlab, I fixed the hyperparameter values estimated by svm function from e1071.

I obtained identical results for the svm model from e1071 and ksvm model from kernlab, but for the implementation of the train function on caret the result was completely different.

Here is a reproducible example:

library(caret)
library(kernlab)
library(tidyverse)
library(e1071)
library(hydroGOF)    
library(AppliedPredictiveModeling)

#data set
data(solubility)
trainData<-cbind(solTrainY,solTrainXtrans)
names(trainData)[1]<-"Y"
testData<-cbind(solTestY,solTestXtrans)
names(testData)[1]<-"Y_holdout"

#e1071 package
SVMr_e1071 <- svm(Y ~ .,  data = trainData, kernel = "radial")

predSVMr_e1071 <- predict(SVMr_e1071, select(testData,-Y_holdout)) 

modelRMSE1<-rmse(testData$Y_holdout, predSVMr_e1071)
modelRMSE1 #0.6535902

#caret package
SVMrGrid <- expand.grid(.C = 1,.sigma=0.004385965)
SVMr_caret <- train(Y ~ .,
                    data=trainData, 
                    method = "svmRadial",
                    tuneGrid = SVMrGrid,
                    trControl = trainControl(method="none",savePredictions=TRUE))

predSVMr_caret <- predict(SVMr_caret,select(testData,-Y_holdout))

modelRMSE2<-rmse(testData$Y_holdout, predSVMr_caret)
modelRMSE2 #1.887908

#kernlab package
SVMr_k<- ksvm(Y ~ .,  data = trainData,kernel ="rbfdot",
               C = 1, kpar = list(sigma=0.004385965))

predSVMr_k <- as.numeric(predict(SVMr_k, select(testData,-Y_holdout))) 

modelRMSE3<-rmse(testData$Y_holdout, predSVMr_k)
modelRMSE3 #0.6535902

I suspect that in the implementation of the train function, the problem is being considered as classification rather than regression. Is there any way to verify if this is in fact happening? Maybe it is a problem related to my R version (R version 3.3.2 (2016-10-31)?

Any help will be much appreciated.

Best regards, Hellen

David Heckmann
  • 2,899
  • 2
  • 20
  • 29
Hellen Geremias
  • 33
  • 6
  • In order to solve your problem we must first recreate it. To recreate it we need the data and the code that produced the same result as you are showing. The code is provided, however the data is lacking. It would be best if you could recreate the behavior with an in built data set. – missuse Mar 28 '18 at 20:59
  • Dear, I add an reproducible example. – Hellen Geremias Mar 29 '18 at 12:29
  • With your example I receive the same RMSE for `kernlab` and `caret` since `caret` calls `kernlab`. While the rmse is different for `e1071` which is not surprising since different hyper parameters were set. Please check the rmse for your example again. – missuse Mar 29 '18 at 12:47
  • when I run the code from your question in a clean R session I receive: `modelRMSE3 #1.234563`, `modelRMSE2 #1.234563` and `modelRMSE1 #0.6535902` which makes sense – missuse Mar 29 '18 at 13:19
  • Thank you for your reply. I will update my R version and try it again to check my results. Best regards, Hellen. By the way, what is your R version? – Hellen Geremias Mar 29 '18 at 13:28
  • R version 3.4.2 (2017-09-28) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200). kernlab_0.9-25, caret_6.0-78, e1071_1.6-8 – missuse Mar 29 '18 at 13:33
  • Dear, I updated my R version (R version 3.4.4 (2018-03-15) Platform: x86_64-w64-mingw32/x64 (64-bit) and installed the packages again. Then I runned the code and now all results are equal (modelRMSE3 #0.6535902, modelRMSE2 #0.6535902 and modelRMSE1 #0.6535902). – Hellen Geremias Mar 29 '18 at 13:47
  • `e1071::ksvm` and `kernlab::ksvm` parameterize the kernel differently. To get equivalent values, you'll need to invert the `svm` value (`SVMrGrid <- expand.grid(C = 1, sigma=1/SVMr_e1071$gamma)` and `kpar = list(sigma=1/SVMr_e1071$gamma)`. When I do this, I get equal results for the 2nd and 3rd models. That said, I get worse RMSE from these models and I don't know why offhand. – topepo Mar 29 '18 at 16:14
  • Thank you topepo for your reply. – Hellen Geremias Apr 02 '18 at 12:07

0 Answers0