2

I have a regression problem, but the cost function is different: The cost for an underestimate is higher than an overestimate. For example, if predicted value < true value, the cost will be 3*(true-predicted)^2; if predicted value > true value, the cost will be 1*(true-predicted)^2.

I'm thinking of using classical regression models such as linear regression, random forest etc. What modifications should I make to adjust for this cost function?

As I know, the ML API such as scikit-learn does not provide the functionality to directly modify the cost function. If I have to use these APIs, what can I do?

Any recommended reading?

W. Yang
  • 49
  • 8

2 Answers2

2

You can use Tensorflow (or theano) for custom cost functions. The common linear regression implementation is here.

To find out how you can implement your custom cost function looking at a huber loss function implemented in tensorflow might help you. Here comes your custom cost function which you should replace in the linked code so instead of 

cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)

in the linked code you'll have:

error = y_known - y_pred
condition = tf.less(error, 0)
overestimation_loss = 1 * tf.square(error)
underestimation_loss = 3 * tf.square(error)    
cost = tf.reduce_mean(tf.where(condition, overestimation_loss, underestimation_loss))

Here when condition is true, error is lower than zero which means you y_known is smaller than y_pred so you'll have overestimation and so the tf.where statement will choose overestimation_loss otherwise underestimation loss.

The secret is that you'll compute both losses and choose where to use them using tf.where and condition.

Update:

If you want to use other libraries, if huber loss is implemented you can take a look to get ideas because huber loss is a conditional loss function similar to yours.

Ash
  • 3,428
  • 1
  • 34
  • 44
0

You can use asymmetric cost function to make your model overestimate or underestimate. You can replace cost function in this implementation with: 

def acost(a): return tf.pow(pred-Y, 2) * tf.pow(tf.sign(pred-Y) + a, 2)

for more detail see this link

Hassaan Saleem
  • 187
  • 1
  • 11