2

I am doing prediction process with SVR and as evaluation metrics I am getting Relative Error (RE)= 42.25% , is it acceptable? Note : I have > 50k instances in my dataset.

Thanks.

Azza Ousji
  • 31
  • 3
  • Is this the mean percent error, the median percent error, the sum of the percent errors, the standard deviation of percent error - that is, what does the 42.25% represent? – James Phillips Nov 15 '18 at 12:03
  • In fact I am not so familiar with the evaluation metrics and I can't differentiate them or interpret them. Using rapid miner the performance operator gives several criterion (ex: RMSE, AE, NAE, RE..) here are what I am getting if you can help : root_mean_squared_error: 55.174 +/- 0.000 absolute_error: 34.705 +/- 42.892 relative_error: 29.31% +/- 90.71% relative_error_lenient: 20.92% +/- 20.02% relative_error_strict: 42.25% +/- 96.76% normalized_absolute_error: 0.821 root_relative_squared_error: 1.063 squared_error: 3044.205 +/- 5279.004 prediction_average: 121.270 +/- 51.884 – Azza Ousji Nov 15 '18 at 12:32
  • What software are you using? – James Phillips Nov 15 '18 at 14:06
  • I amusing RapidMiner – Azza Ousji Nov 15 '18 at 14:10

1 Answers1

3

As always when judging the quality of a model: it depends. It depends on your data, on your goal, on your "costs" for errors...

What you can see from your different metrics is mainly, that you have a huge variance in performance throughout your predictions. So a relative error of ~42% is meaningless, if you consider the +/-97% variance.

Looking at your absolute error, you miss your goal by ~35 "units" on average, but with a variance of +/-43.

For me, all these metrics scream "we're not dependable", which in turn points to an unsuitable model.

Christian König
  • 3,437
  • 16
  • 28