Stack Overflow
Stack Overflow

Questions tagged [dropout]

Dropout is a technique to reduce overfitting during the training phase of a neural network.

Dropout is a regularization technique for reducing overfitting in neural networks by preventing complex co-adaptations on training data. The term "dropout" refers to dropping out units (both hidden and visible) in a neural network.

215 questions
0
votes
1 answer

Running time increasing from ~10 seconds per epoch to ~120 seconds per epoch after adding Dropout?

I'm training a simple neural network that looks as follows; model = Sequential() model.add(layers.GRU(32, input_shape=(None, 13))) model.add(layers.Dense(1)) model.compile(optimizer=RMSprop(), loss='mae') history =…
Psychotechnopath
  • 2,471
  • 5
  • 26
  • 47
0
votes
2 answers

CNN having high overfitting despite having dropout layers?

For some background, my dataset is roughly 75000+ images, 200x200 greyscale, with 26 classes (the letters of the alphabet). My model is: model = Sequential() model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(200, 200, 1)))…
JupyterBoi
  • 39
  • 1
  • 6
0
votes
1 answer

How to avoid excessive memory usage while training multiple models in Tensorflow

I'm currently writing a piece of code that aims to explain how applying varying dropout rates influence the performance of a generic CNN model across several datasets. I've set it up so, for each dataset, I train 10 different models (with 10…
Miguel Mano
  • 103
  • 1
  • 12
0
votes
0 answers

Add dropout to Keras Model

I would like to add a dropout in hidden layers. I do not know if I should implement the sentence after or before of the definition of each hidden layer. May somebody help me? def keras_model2(layers_dims): L = len(layers_dims) model =…
Miguel Santos
  • 134
  • 8
0
votes
1 answer

Why do we need to preseve the "expected output" during dropout?

I am very confused as to why do we need to preserve the value of the expected output when performing dropout regularisation. Why does it matter if the mean of the outputs of layer l is different in the training and testing phase? Weights that are…
Devanshi Sukhija
  • 97
  • 5
0
votes
2 answers

Keras minibatch gradient descent with Dropout layer

I have a question about Dropout implementation in Keras/Tensorflow with mini-batch gradient descent optimization when batch_size parameter is bigger than one. The original paper says: The only difference is that for each training case in a…
kiku06
  • 1
  • 2
0
votes
2 answers

How to turn off droput during validation/tesing phase?

I'm new in neural network. I know that during the validation/testing time the dropout must be to turn off because dropout makes neurons output 'wrong' values on purpose. So it is better in order to have a good result in term of accuracy. How can I…
Gabriele Valvo
  • 196
  • 2
  • 12
0
votes
0 answers

Dropouts in AWD-LSTM

I am trying to implement AWD-LSTM and thus would like to ensure I understand the dropout techniques correctly. I have read the article & fastai docs but still doubt whether I have understood it properly. Embedding dropout (embed_p) - probability of…
Akim Tsvigun
  • 91
  • 1
  • 8
0
votes
1 answer

pytorch 1D Dropout leads to unstable learning

I'm implementing an Inception-like CNN in pytorch. After the blocks of convolution layers, I have three fully-connected linear layers followed by a sigmoid activation to give me my final regression output. I'm testing the effects of dropout layers…
GeneC
  • 145
  • 2
  • 10
0
votes
0 answers

I don't understand how Keras behaves when Dropout exists

I built a Neural Network with Dropout using Keras in Python. I want to find out how this network updates the weights. For simplicity, only one training data was prepared. I trained the model under the following conditions: (1): Model that doesn't…
Tine
  • 9
  • 3
0
votes
1 answer

Restoring saved model with dropout applied from java program

I have a pre-trained model with dropout applied and I want to restore it from java program. With my application, in the inference step, I need to turn on dropout and repeat feeding input to the model multiple times and get an array of predictions.…
quangbk2010
  • 158
  • 1
  • 6
0
votes
1 answer

R Keras: apply dropout regularization both on the input and the hidden layer

I'm learning Keras in R and want to apply dropout regularization both on the input layer as it's very big (20000 variables) and the intermediate layer (100 neurons). I'm deploying Keras for regression. From the official documentation I've reached to…
nba2020
  • 618
  • 1
  • 8
  • 22
0
votes
0 answers

How does dropout work (with multiple GPUs)?

Let's say I'm using multiple GPUs and I'm training a neural network that's using dropout. I know that dropout randomly turns off certain nodes in the network for each training sample, and then only updates the weights in the "thinned network," then…
Jonathan
  • 1,876
  • 2
  • 20
  • 56
0
votes
1 answer

Cannot restore Dropout using get_tensor_by_name

I tried to save and restore some tensors. In saving session: ... self.abc = tf.reduce_sum(self.element_wise_product, 2, name="abc") self.def= tf.nn.dropout(abc, self.dropout_keep[0], name="def") ... After saving, I tried to restore the…
T D Nguyen
  • 7,054
  • 4
  • 51
  • 71
0
votes
1 answer

How to implement dropout with tensorflow

I have applied dropout in the tensorflow 3-knn implementation. But I have got an error due to the placeholder's variable keep_prob. TypeError: Cannot interpret feed_dict key as Tensor: Can not convert an int into a Tensor. I have written 2…
Willy Zart
  • 3
  • 3
1 2 3
…
14 15