3

When I do a classification job, I need to encode a classid with one_hot method. But shuold I encode background class with -1 or 0 with tf.one_hot function?

For example:

// plan a
logits = [0.1, 0.1, 0.2, 0.3, 0.4]
classids = [-1,1,2,3,4] // -1 is background class
class_num = 5
on_hot_class = tf.one_hot(class_ids, depth=class_num)
loss = tf.keras.losses.categorical_crossentropy(one_hot_class,class_logits, from_logits=True)

// plan b 
logits = [0.1, 0.1, 0.2, 0.3, 0.4]
classids = [0,1,2,3,4] // 0 is background class
class_num = 5
on_hot_class = tf.one_hot(class_ids, depth=class_num)
loss = tf.keras.losses.categorical_crossentropy(one_hot_class,class_logits, from_logits=True)
tidy
  • 4,747
  • 9
  • 49
  • 89
  • 1
    Here are a couple of related questions where I posted similar answers: [How to create a class for non classified object in tensorflow?](https://stackoverflow.com/q/47013114/1782792), [Rule of thumb for negative example in multi-class classification](https://stackoverflow.com/q/48198306/1782792). – jdehesa May 17 '19 at 13:20

1 Answers1

3

Canonically, you would treat your background class just like any other and encode as a one_hot(on_value=1). If you want to emphasize this class you can use weighted tf.nn.weighted_cross_entropy_with_logits and assign a higher weight to that class.

Since you rely on crossentropy with logits, the output of logits function would always be between 0 and 1. It means, that your model would always produce high loss value whenever it sees an input of background class. It is very likely to disrupt your training. It still does not mean that you can not use -1 or that it would have no positive effect in any imaginable situation.

y.selivonchyk
  • 8,987
  • 8
  • 54
  • 77