Stack Overflow
Stack Overflow

Questions tagged [dropout]

Dropout is a technique to reduce overfitting during the training phase of a neural network.

Dropout is a regularization technique for reducing overfitting in neural networks by preventing complex co-adaptations on training data. The term "dropout" refers to dropping out units (both hidden and visible) in a neural network.

215 questions
4
votes
2 answers

Is dropout layer still active in a freezed Keras model (i.e. trainable=False)?

I have two trained models (model_A and model_B), and both of them have dropout layers. I have freezed model_A and model_B and merged them with a new dense layer to get model_AB (but I have not removed model_A's and model_B's dropout layers).…
陳建勤
  • 127
  • 6
4
votes
2 answers

Dropout layer directly in tensorflow: how to train?

After I created my model in Keras, I want to get the gradients and apply them directly in Tensorflow with the tf.train.AdamOptimizer class. However, since I am using a Dropout layer, I don't know how to tell to the model whether it is in the…
John Brown
  • 65
  • 1
  • 7
4
votes
2 answers

Problem with Dropout version Google Colab

I am trying to add a dropout layer and I get this warning WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and…
Javier Naranjo Alcázar
  • 105
  • 6
4
votes
2 answers

How to use MC Dropout on a variational dropout LSTM layer on keras?

I'm currently trying to set up a (LSTM) recurrent neural network with Keras (tensorflow backend). I would like to use variational dropout with MC Dropout on it. I believe that variational dropout is already implemented with the option…
Nosk
  • 753
  • 2
  • 6
  • 24
4
votes
1 answer

Keras Dropout Layer Model Predict

The dropout layer is only supposed to be used during the training of the model, not during testing. If I have a dropout layer in my Keras sequential model, do I need to do something to remove or silence it before I do model.predict()?
c19ut
  • 59
  • 4
4
votes
1 answer

About correctly using dropout in RNNs (Keras)

I am confused between how to correctly use dropout with RNN in keras, specifically with GRU units. The keras documentation refers to this paper (https://arxiv.org/abs/1512.05287) and I understand that same dropout mask should be used for all…
tu_curious
  • 387
  • 4
  • 13
4
votes
1 answer

Monte Carlo (MC) dropout in Keras with R

How to implement Monte Carlo dropout with Keras in Convolutional neural networks to estimate predictive uncertainty as suggested by YARIN GAL? I am using R.R-Code is here I am fitting the model in small batches and want to evaluate the model in…
Ehtasham Billah Mymun
  • 101
  • 7
4
votes
0 answers

Tensorflow Language Model tutorial dropout twice?

I'm working on the Language Model tutorial of Tensorflow. My question is: In this line, they use a Wrapper to apply dropout to the RNNs lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(size, forget_bias=0.0) if is_training and config.keep_prob < 1: …
tnq177
  • 583
  • 1
  • 8
  • 19
3
votes
1 answer

" ValueError: Expecting KerasTensor which is from tf.keras.Input()". Error in prediction with dropout function

I am trying to predict uncertainty in a regression problem using Dropout during testing as per Yarin Gal's article. I created a class using Keras's backend function as provided by this stack overflow question's answer. The class takes a NN model…
Anshul Nayak
  • 41
  • 1
  • 5
3
votes
1 answer

Batchnormalize, Dropout and number of layers

I'm learning batchnormalisation and dropout. Saw this https://www.kaggle.com/ryanholbrook/dropout-and-batch-normalization. The model model = keras.Sequential([ layers.Dense(1024, activation='relu', input_shape=[11]), layers.Dropout(0.3), …
Data T
  • 117
  • 1
  • 8
3
votes
1 answer

Should my model with Monte Carlo dropout provide a mean prediction similar to the deterministic prediction?

I have a model trained with multiple LayerNormalization layers, and I am unsure if a simple weight transfer works properly when activating dropout for prediction. This is the code I am using: from tensorflow.keras.models import load_model,…
WVJoe
  • 515
  • 7
  • 21
3
votes
1 answer

GaussianDropout vs. Dropout vs. GaussianNoise in Keras

Can anyone explain the difference between the different dropout styles? From the documentation, I assumed that instead of dropping some units to zero (dropout), GaussianDropout multiplies those units by some distribution. However, when testing in…
JP K.
  • 1,231
  • 1
  • 9
  • 12
3
votes
1 answer

PyTorch: Dropout (?) causes different model convergence for training+validation V. training-only

We are facing a very strange issue. We tested the exact same model into two different “execution” settings. In the first case, given a certain amount of epochs, we train using mini-batches for one epoch, and thereafter we test on the validation set…
Andrea
  • 151
  • 1
  • 10
3
votes
2 answers

What layers are affected by dropout layer in Tensorflow?

Consider transfer learning in order to use a pretrained model in keras/tensorflow. For each old layer, trained parameter is set to false so that its weights are not updated during training whereas the last layer(s) have been substituted with new…
roschach
  • 8,390
  • 14
  • 74
  • 124
3
votes
1 answer

Why setting training=True in tf.keras.layers.Dropout during testing mode is leading to lower training loss values and higher prediction accuracy?

I'm using dropout layers on my model implemented in tensorflow (tf.keras.layers.Dropout). I set the "training= True" during the training and "training=False" while testing. The performance is poor. I accidentally changed "training=True" during…
khemedi
  • 774
  • 3
  • 9
  • 19
1 2
3
…
14 15