Stack Overflow
Stack Overflow

Questions tagged [dropout]

Dropout is a technique to reduce overfitting during the training phase of a neural network.

Dropout is a regularization technique for reducing overfitting in neural networks by preventing complex co-adaptations on training data. The term "dropout" refers to dropping out units (both hidden and visible) in a neural network.

215 questions
0
votes
0 answers

tf.nn.dropout when used outputs same value

When i don't use dropout in my model for cat and dog classification, the predicted values remain normal i.e not the same value for all images. But when I use tf.nn.dropout with keep_prob = 0.8 for my model which was recommended for regularizing the…
Arif Raihan
  • 1
0
votes
1 answer

Random dropout rates for tf.nn.dropout

I am experimenting with the TensorFlow dropout function. Since a functionality where the dropout rate decreases with time during training was too complicated to implement (tried all day yesterday), I thought using random dropout rates for each…
beinando
  • 477
  • 4
  • 18
0
votes
2 answers

How to create a CUMULATIVE dropout rate table from raw data

I'm trying to modify a solution posted here Create cohort dropout rate table from raw data I'd like to create a CUMULATIVE dropout rate table using these data. DT<-data.table( id =c (1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17,18,19,20, …
Chris
  • 353
  • 3
  • 9
0
votes
1 answer

Periodical pattern in loss function in a convolutional neural network (tensorflow)

I'm working on image segmentation using a Convolutional Neural Network (cnn) implemented in Tensorflow. I have two classes and I am using cross entropy as loss function and as Adam optimizer. I am training the network with around 150 images. During…
Decano
  • 93
  • 6
0
votes
1 answer

How to apply dropout to the outputs of an RNN in TensorFlow Eager using the Keras API?

I would like to apply dropout to the outputs from an RNN. For example, in Tensorflow 1.8.0, I could do this: import tensorflow as tf import tensorflow.contrib.eager as tfe tfe.enable_eager_execution() x = tf.random_uniform((10, 5, 3)) gru_cell1 =…
mauna
  • 1,098
  • 13
  • 25
0
votes
1 answer

Dropout rate in bottle neck layers

It is common to use a dropout rate of 0.5 as a default which I also use in my fully-connected network. This advise follows the recommendations from the original Dropout paper (Hinton at al). My network consists of fully-connected layers of…
Gilfoyle
  • 3,282
  • 3
  • 47
  • 83
0
votes
1 answer

Which kind of regularization use L2 regularization or dropout in multiRNNCell?

I have been working on a project related with sequence to sequence autoencoder for time series forecasting. So, I have used tf.contrib.rnn.MultiRNNCell in encoder and decoder. I am confused in which strategy used in order to regularize my seq2seq…
dnovai
  • 137
  • 1
  • 8
0
votes
1 answer

LSTM Dropout Wrapper Rank Error

I have an LSTM model which works "correctly" when specifying my dropout keep rate as follows: layers = [tf.contrib.rnn.DropoutWrapper(tf.contrib.rnn.BasicLSTMCell(num_units=n_neurons, activation=tf.nn.tanh), output_keep_prob=0.5) for layer in…
The Rhyno
  • 103
  • 7
0
votes
1 answer

Static Dropout Layer - Keras

I am interested in making a "dropout like" layer that is static throughout the course of training and testing, in keras. Unlike normal dropout, I only want to sever a certain amount of random weights, not the entire node. I.e. certain neurons cannot…
Jordan Ott
  • 31
  • 3
0
votes
1 answer

Tensorflow Dropout: What happens if I apply two dropout layers?

Lets say, I am constructing a neural network like so: x = tf.nn.conv2d(input, ...) x = tf.nn.max_pool(x, ...) x = tf.nn.dropout(x, keep_prob=1.) x = tf.nn.thislayershallhavedropout(x,...) x = tf.nn.dropout(x, keep_prob=.5) Would this be an…
Ulu83
  • 523
  • 9
  • 20
0
votes
1 answer

dropout and data.split in model.fit

As we know, dropout is a mechanism to help control overfitting. In the training process of Keras, we can conduct online cross-validation by monitoring the validation loss, and setup data split in model.fit. Generally, do I need to use both of these…
user297850
  • 7,705
  • 17
  • 54
  • 76
-1
votes
1 answer

where to add dropout layer in the model?

from my understanding the use of dropout regularization randomly disables some portion of neurons in a hidden layer and it helps with the overfitting problem I am using this implementation of resnet with pytorch and I want to add a dropout…
amy
  • 1
  • 1
-1
votes
1 answer

Should feature embeddings be taken before or after dropout layer in neural network?

I am training a binary text classification model using BERT as follows: def create_model(): text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text') preprocessed_text = bert_preprocess(text_input) outputs =…
Jane Sully
  • 3,137
  • 10
  • 48
  • 87
-1
votes
1 answer

What happens if my Dropout is too high? what Dropout to use on my 2048-neuron-dense layer? (very little data)

I am pretty new to this and I am writing my bachelor thesis in keras. I have this big CNN, built similar to vgg but a bit different, because I have bigger resolution images and I pool a little more. I added a 2048 dense layer on top. What Dropout do…
Shaved Man
  • 57
  • 1
  • 10
-1
votes
2 answers

Why we must drop out in Tensorflow?

I have read this article, https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout. Dropout will help prevent overfitting by make non-active neutron in ANN. But the next question... why we must dropout neutron since we can adjust how much…
Ichsan
  • 768
  • 8
  • 12
1 2 3
…
14
15