How to add noise to activations at in Keras at inference time?

Question

GaussianNoise in Keras seems to be only to add noise during training time. I need to add noise to activations in test time. My architecture is resnet50 pretrained on imagenet with all layers frozen, except for the fact that the gaussian noise needs to be added to the last activation layer prior to the FC layer.

How can this be done!? The gaussian noise layer I added as below at the end is not making any effect as the documentation says its only for during training. What is the alternative to this layer during test time?

__________________________________________________________________________________________________
bn5c_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_80 (Add)                    (None, 7, 7, 2048)   0           bn5c_branch2c[0][0]              
                                                                 activation_242[0][0]             
__________________________________________________________________________________________________
activation_245 (Activation)     (None, 7, 7, 2048)   0           add_80[0][0]                     
__________________________________________________________________________________________________
gaussian_noise_1519 (GaussianNo (None, 7, 7, 2048)   0           activation_245[0][0]             
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 2048)         0           gaussian_noise_1519[0][0]        
__________________________________________________________________________________________________
fc1000 (Dense)                  (None, 1000)         2049000     avg_pool[19][0]                  
==================================================================================================
Total params: 25,636,712
Trainable params: 0
Non-trainable params: 25,636,712

am running some specific robustness tests and implementing some research ideas as well — hearse, Jul 31 '19 at 05:02
Can't you use do the following? 1. Define a model (with/without noise) for training 2. Define an inference model that uses a `Lambda` layer which adds noise 3. Copy the trained weights from the trained model to the inference model — thushv89, Jul 31 '19 at 05:25
I'm new to keras. Can you provide detailed code in answer instead? — hearse, Jul 31 '19 at 05:33

score 3 · Answer 1 · answered Jul 31 '19 at 05:34

3

You can keep active those layers which have different behavior in test phase (e.g. Dropout) by passing training=True argument when calling them on a tensor:

out = SomeLayer(**configs)(inp, training=True)

With that, SomeLayer would be active in both training and test phases.

answered Jul 31 '19 at 05:34

today

32,602
8
95
115

nope because https://keras.io/layers/noise/ says it is only active at training time regardless and doesnt work for test time!! – hearse Jul 31 '19 at 05:36
2

@hearse That's the normal case (i.e. without passing `training` argument). Please read answers carefully. And also the documentation: nowhere in the doc says that with `training=True` the guassian noise layer would be inactive. – today Jul 31 '19 at 05:37
1

@hearse And the irony is that you say "I am new to Keras" and then express such confident claims about it. – today Jul 31 '19 at 05:42
can I do without having to train the entire imagenet? just need to add simple additive gaussian noise at inference time after 1 particular layer. Retraining all of imagenet seems such a long route – hearse Jul 31 '19 at 05:43

How to add noise to activations at in Keras at inference time?

1 Answers1