Getting the output prior to non-linear activation in Keras

Question

How can I get the value prior to activation when I use the following syntax to define a layer in Keras:

model.add(Convolution2D(128, 5, 5, activation='relu'))

I know that I can simply use:

model.add(Convolution2D(128, 5, 5))
model.add(Activation('relu'))

and get the output from the first layer, but is it possible when using the first syntax?

Answer 1

No, you can't do this in a simple way since the activation function is applied right after getting the output of convolution in the implementation of convolution layer in Keras:

if self.rank == 2:
    outputs = K.conv2d(
        inputs,
        self.kernel,
        strides=self.strides,
        padding=self.padding,
        data_format=self.data_format,
        dilation_rate=self.dilation_rate)

if self.activation is not None:
    return self.activation(outputs)
return outputs

Though, you might be able to write a custom code to perform convolution and define a custom function to get the raw outputs of convolution. Another option is to write your own convolution layer (don't worry, it is so easy to do this!) which has two outputs: the result of applying convolution and also the result of applying activation function.