13

I am trying to train a model using SciKit Learn's SVM module. For the scoring, I could not find the mean_absolute_error(MAE), however, negative_mean_absolute_error(NMAE) does exist. What is the difference between these 2 metrics? Lets say I get the following results for 2 models:

model 1 (NMAE = -2.6), model 2(NMAE = -3.0)

Which model is better? Is it model 1?

Moreover, how does the negative compare to the positive? Say the following:

model 1 (NMAE = -1.7), model 2(MAE = 1.4)

Here, which model is better?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
darkhorse
  • 8,192
  • 21
  • 72
  • 148

2 Answers2

20

As its name implies, negative MAE is simply the negative of the MAE, which (MAE) is by definition a positive quantity. And since MAE is an error metric, i.e. the lower the better, negative MAE is the opposite: a value of -2.6 is better than a value of -3.0.

Just remove the negative signs and treat them as MAE values (which arguably also answers your second question).

Keep in mind that MAE is always available in scikit-learn as a general metric (docs).

desertnaut
  • 57,590
  • 26
  • 140
  • 166
12

I would like to add here, that this negative error is also helpful in finding best algorithm when you are comparing multiple algorithms through GridSearchCV().

This is because after training, GridSearchCV() ranks all the algorithms(estimators) and tells you which one is the best. Now when you use an error function, estimator with higher score will be ranked higher by sklearn, which is not true in the case of MAE (along with MSE and a few others).

To deal with this, the library flips the sign of error, so the highest MAE will be ranked lowest and vice versa.

So to answer your question: -2.6 is better than -3.0 because the actual MAE is 2.6 and 3.0 respectively.

Eushay Rana
  • 156
  • 1
  • 5