Question about enabling/disabling dropout with keras functional API

Question

I am using Keras functional API to build a classifier and I am using the training flag in the dropout layer to enable dropout when predicting new instances (in order to get an estimate of the uncertainty). In order to get the expected response one needs to repeat this prediction several times, with keras randomly activating links in the dense layer, and of course it is computational expensive. Therefore, I would also like to have the option to not use dropout at the prediction phase, i.e., use all the network links. Does anyone know how I can do this? Following is a sample code of what I am doing. I tried to look if predict has any relevant parameter but does not seem like it does (?). I can technically train the same model without the training flag at the dropout layer, but I do not want to do this (or better I want to have a more clean solution, rather than having 2 different models).

from sklearn.datasets import make_circles
from keras.models import Sequential
from keras.utils import to_categorical
from keras.layers import Dense
from keras.layers import Dropout
import numpy as np
import keras

# generate a 2d classification sample dataset
X, y = make_circles(n_samples=100, noise=0.1, random_state=1)
n_train = 30
trainX, testX = X[:n_train, :], X[n_train:, :]
trainy, testy = y[:n_train], y[n_train:]
trainy = to_categorical(trainy)
testy = to_categorical(testy)

inputlayer = keras.layers.Input((2,))

d = keras.layers.Dense(500, activation = 'relu')(inputlayer)
d1 = keras.layers.Dropout(rate = .3)(d,training = True)
out = keras.layers.Dense(2, activation = 'softmax')(d1)

model = keras.Model(inputs = inputlayer, outputs = out)

model.compile(loss = 'categorical_crossentropy',metrics = ['accuracy'],optimizer='adam')
model.fit(x = trainX, y = trainy, validation_data=(testX, testy),epochs=1000, verbose=1)

# another prediction on a specific sample 
print(model.predict(testX[0:1,:]))
# another prediction on the same sample
print(model.predict(testX[0:1,:]))

Running the above example I get the following output:

[[0.9230819  0.07691813]]
[[0.8222245  0.17777553]]

which is as expected, different class probabilities for the same input, since there is a random (de)activation of the links from the dropout layer.

Any suggestions on how I can enable/disable dropout at the prediction phase with the functional API?

Possible duplicate of [How to deactivate a dropout layer called with training=True in a Keras model?](https://stackoverflow.com/questions/57437941/how-to-deactivate-a-dropout-layer-called-with-training-true-in-a-keras-model) — today, Aug 18 '19 at 20:21

score 0 · Answer 1 · answered Aug 18 '19 at 18:47

Sure, you do not need to set the training flag when building the Dropout layer. After training your model you define this function:

mc_func = K.function([model.input, K.learning_phase()],
                     [model.output])

Then you call mc_func with your input and flag 1 to enable dropout, or 0 to disable it:

stochastic_pred = mc_func([some_input, 1])
deterministic_pred = mc_func([some_input, 0])

Question about enabling/disabling dropout with keras functional API

1 Answers1