How to obtain F1, precision, recall and confusion matrix

Question

the goal of my project is to predict the accuracy level of some textual descriptions.

I made the vectors with FASTTEXT.

TSV output:

 label lenght
1     0   1:43
2     0   1:10
3     0    1:8
4     0  1:110
5     1  1:105
6     0  1:446

Then I'm processing the .tsv files using the library e1071 by this script:


library (e1071)

accuracy.model = read.table(file = 'file.tsv', sep = '\t', header = FALSE, col.names= c( "label", "lenght" ))

head(accuracy.model)

classifier = svm( formula = label ~ .,
                  data = accuracy.model,
                  type = 'C-classification',
                  kernel = 'radial',
                  cost = 32,
                  gamma = 8,
                  cross  = 10)

classifier

Doing the cross validation (10 folds) I'm able to retrieve the overall accuracy percentage.

I would like to have also the F1 score, precision and recall value.

For the confusion matrix I went through some others stack thread and I figured out that has to be done using the caret library but I don't know how to do it.

Suggestions?

Regards

A note of caution, i noticed you have only 1 independent variable, which is really odd for using an svm. You might wanna check whether this is sound — StupidWolf, Apr 08 '20 at 11:34

score 1 · Accepted Answer · answered Apr 08 '20 at 11:33

Let's say we fit a model like this:

library(caret)
library(e1071)
data=iris
data$Species = ifelse(data$Species=="versicolor","v","o")

classifier = svm( formula = Species ~ .,
                  data = data,
                  type = 'C-classification',
                  kernel = 'radial',
                  cost = 32,
                  gamma = 8,
                  cross  = 10)

Then we get the confusion matrix:

mat = table(classifier$fitted,data$Species)

And apply the caret function:

confusionMatrix(mat)$byClass

         Sensitivity          Specificity       Pos Pred Value 
           1.0000000            1.0000000            1.0000000 
      Neg Pred Value            Precision               Recall 
           1.0000000            1.0000000            1.0000000 
                  F1           Prevalence       Detection Rate 
           1.0000000            0.6666667            0.6666667 
Detection Prevalence    Balanced Accuracy 
           0.6666667            1.0000000

You can apply it in your case.

How to obtain F1, precision, recall and confusion matrix

1 Answers1