2

I'm doing multiclass-multilabel classification. Namely, I have N_labels fully independent labels for each example, whereas each label may have N_classes different values (mutually exclusive). More concretely, each example is classified by N_labels-dimensional vector, while each vector components can by from the set {0, 1, ..., N_classes}

For example, if N_labels = 5 and N_classes = 3, each example may be classified by the following tags:

[2, 1, 0, 0, 1], [0, 0, 2, 2, 1], [0, 0, 0, 0, 0]

In addition, for each label I have very imbalance between different classes, namely 90% of examples in training set belong to set 0. So, I'd like to perform weighted softmax cross entropy in order to compute loss for each label (and average afterwards).

Tried to use: 

tf.losses.sparse_softmax_cross_entropy # but it seems that it performs weightening between different label and not between classes for each label.

tf.nn.softmax_cross_entropy_with_logits,  tf.nn.softmax_cross_entropy_with_logits_v2  # does not have weightening option ever

tf.nn.weighted_cross_entropy_with_logits  # good only for binary classification

I'd like to find compute_loss function to compute loss in the following way:

loss = compute_loss(logits=my_logits, labels=my_labels, weights=my_weights)

where 

my_logits is of shape [batch_size, N_labels, N_classes]
my_labels is of shape [batch_size, N_labels]
my_weight is of shape [N_labels, N_classes]

Note that each label may have different weights (for classes)

Mike E
  • 43
  • 7
  • 1
    Can you provide an input `logits`, `labels`, `weights` and a desired output? – gorjan Mar 26 '19 at 10:44
  • Hi Mike, welcome to stackoverflow. You mention that for each Label your Classes are mutually exclusive. I am not sure I get this from the example you provide. – CAPSLOCK Mar 26 '19 at 10:44
  • @Gio, labels are not mutually exclusive. Each example is ALWAYS labeled with, say, 5 labels. And each label may have 3 different values (e.g. 0, 1, 2) – Mike E Mar 26 '19 at 10:49
  • @gorjan, my input dimensions are: my_logits.shape = [batch_size, N_labels, N_classes] my_labels.shape = [batch_size, N_labels] my_weight.shape = [N_labels, N_classes]. Logits are real numbers, my_weight are positive numbers with sum(my_weight, axis=1) = ones(N_labels), while each component of my_labels may be each number from {0, 1, ..., N_classes} – Mike E Mar 26 '19 at 10:49

1 Answers1

1

I think you need tf.losses.sigmoid_cross_entropy It uses multi_class_labels just as you described, and have functionality to apply weights. https://www.tensorflow.org/api_docs/python/tf/losses/sigmoid_cross_entropy

Example: Suppose you have a multiclass multilabel classification problem, where you have 10 classes in total and label for single example look like this [1, 3, 6], meaning example contains classes 1, 3 and 6.
You need to use k-hot encoding 

labels = tf.reduce_max(tf.one_hot([1, 3, 6], 10, dtype=tf.int32), axis=0)

In this case output will be [0, 1, 0, 1, 0, 0, 1, 0, 0, 0]

Sharky
  • 4,473
  • 2
  • 19
  • 27
  • not exactly. It is designed to solely multiclass classification when it is not combined with multilabel. But It can be used for each label separately and this is the way I doing it now invoking tf.losses.sparse_softmax_cross_entropy() . – Mike E Mar 27 '19 at 09:24
  • 1
    No, this actually the function for multiclass multilabel classification. Added example – Sharky Mar 27 '19 at 13:25
  • If the answer was helpful, please consider accepting it https://stackoverflow.com/help/someone-answers – Sharky Mar 28 '19 at 14:13