0

So I have 100x100x3 images and its a classification problem with 3 categories.

So my CNN architecture is as follows:

visible = Input(shape=(100, 100, 3))
conv_1 = Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), padding='same')(visible)
flatten_1 = Flatten()(conv_1)
dense_1 = Dense(100)(flatten_1)
dense_2 = Dense(3)(dense_1)
act_6 = Activation('softmax')(dense_2)
model = Model(inputs=visible, outputs=act_6)

Now I define my custom loss function as follows:

def custom_loss(visible):
    def loss_fun(y_true, y_pred):
        return sparse_categorical_crossentropy(y_true, y_pred) + visible[?, 0, 0, 0]
    return loss_fun

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss=custom_loss(visible), optimizer=sgd)

Of course, the above code gives an error because of "?".
As you can see, I want to add the pixel value of the image which is at (0, 0, 0) for every image to the loss function.
In fact I want to be able to change the (0, 0, 0) to some (x, y, z) depending upon my y_pred for that particular image.
How can I do that?

idpd15
  • 448
  • 2
  • 5
  • 22
  • Which TF version are you using ? – thushv89 Jan 09 '20 at 07:29
  • And what do you mean by "In fact I want to be able to change the (0, 0, 0) to some (x, y, z) depending upon my y_pred for that particular image."? How exactly do you want to change? – thushv89 Jan 09 '20 at 07:33
  • @thushv89 For practical purposes, you can assume (x, y, z) = f(y_pred) . Tf version is 1.14 – idpd15 Jan 09 '20 at 07:40
  • Well that's not very helpful. You need to provide a concrete function for `f` so that we can test it before posting the answer. – thushv89 Jan 09 '20 at 07:41
  • @thushv89 you can assume any function `f` for your testing. Eg if y_pred == 0: (x, y, z) = (0, 1, 2) elif y_pred == 1: (x, y, z) = (1, 3, 5) else: (x, y, z) = (7, 8, 8) – idpd15 Jan 09 '20 at 07:45
  • SO y_pred is a `(None, 3)` sized output, and are real numbers did you mean the argmax value of y_pred? – thushv89 Jan 09 '20 at 08:33
  • @thushv89 yes.. – idpd15 Jan 09 '20 at 08:59

1 Answers1

1

You can do the following,

def custom_loss(visible):
    def loss_fun(y_true, y_pred):
        # Getting the classes
        classes = tf.argmax(y_pred, axis=1)
        # The different x,y,z combinations for each class (None, 3) sized array
        # We are doing a gather operation to select correct combination of x,y,z for a given class
        inds = tf.gather(tf.constant([[0,1,2],[1,3,1],[7,8,1]], dtype=tf.int32), classes)
        # We concatenate the batch dimnsion to the front [0, 1, 2, ....]
        final_inds = tf.concat([tf.cumsum(tf.ones_like(classes, dtype=tf.int32))[:,tf.newaxis], inds], axis=1)

        # Get the related ids from visible
        return sparse_categorical_crossentropy(y_true, y_pred) + tf.reduce_mean(tf.gather_nd(visible,final_inds))
    return loss_fun

PS: Bear in mind that this second part of the loss (coming from visible) is not differentiable (because it is just an input and does not have any parameters).

thushv89
  • 10,865
  • 1
  • 26
  • 39