Stack Overflow
Stack Overflow

Questions tagged [dropout]

Dropout is a technique to reduce overfitting during the training phase of a neural network.

Dropout is a regularization technique for reducing overfitting in neural networks by preventing complex co-adaptations on training data. The term "dropout" refers to dropping out units (both hidden and visible) in a neural network.

215 questions
0
votes
1 answer

Question about enabling/disabling dropout with keras functional API

I am using Keras functional API to build a classifier and I am using the training flag in the dropout layer to enable dropout when predicting new instances (in order to get an estimate of the uncertainty). In order to get the expected response one…
user1294691
  • 13
  • 2
0
votes
2 answers

Is it a good model in the making with validation loss remaining lower than train curve, more or less constant gap as both are decreasing?

Adding dropout layers made the val loss remain lower than train loss, is it exceptable to have a constant generalization gap over the period? Here is the architecture: tf.keras.layers.CuDNNLSTM(1024,input_shape=(9,41),return_sequences=True)…
Srinath
  • 19
  • 4
0
votes
0 answers

TOCO failed ConverterError for converting .pb to .tflite keras

I am loading a keras model from json file like this: with open(str(incoming_json_file),'r') as fb: con = json.load(fb) where my keras model is defined like this: { "model": "Sequential", "layers": [ { "L1":…
Ashutosh Mishra
  • 183
  • 1
  • 1
  • 10
0
votes
1 answer

Using Dropout on Convolutional Layers in Keras

I have implemented a convolutional neural network with batch normalization on 1D input signal. My model has a pretty good accuracy of ~80%. Here is the order of my layers: (Conv1D, Batch, ReLU, MaxPooling) repeat 6 times, Conv1D, Batch, ReLU, Dense,…
Alex Charoen
  • 11
  • 1
  • 1
0
votes
1 answer

Custom Layers in tensorflow

I am trying to make some changes to the inbuilt dropout function in tensorflow. What is the best procedure to do so? I'd like to make some changes in forward and backpropogation steps. In Tensorflow Implementation I can only find forward pass not…
Terwayp
  • 51
  • 4
0
votes
0 answers

What is the proper way to implement an inverse dropout layer in keras?

I read a lot of math explanations on this here which I understood nothing. Can anyone provide a simple code example? I am looking for a keras sequential model example. Thanks for all the answers.
Leon Rai
  • 1,401
  • 3
  • 9
  • 15
0
votes
1 answer

How can I tell Keras the learning phase when I use train_on_batch to train a model?

I have dropout layers in my model so I want keras to figure out the training and test phases to run or ignore the dropout layers, and I found that K.set_learning_phase can do me this favor but how can I add it to training and test processes? My code…
Yutas
  • 3
  • 4
0
votes
2 answers

Input contains NaN, infinity or a value too large for dtype('float64') in Tensorflow

I am trying to train a LSTM and in my model I have an exponential learning rate decay and a dropout layer. In order to deactivate the dropout layer when testing and validating, I have put a placeholder for the dropout rate and given it a default…
Suleka_28
  • 2,761
  • 4
  • 27
  • 43
0
votes
1 answer

How add dropout into my tensorflow neural network with RNNCells?

I have some neural network (tensorflow) n_steps = 10 n_inputs = 3 n_outputs = 1 n_neurons = 100 n_layers = 3 X = tf.placeholder(tf.float32, [None, n_steps, n_inputs]) y = tf.placeholder(tf.float32, [None, n_steps,…
user10620635
  • 27
  • 7
0
votes
1 answer

The understanding about dropout in DNN

From what I understand about DNN's dropout regularization is that: Dropout: First we randomly delete neurons from the DNN and leave only the input and output the same. Then we perform forward propagation and backward propagation based on a…
jjsonname
  • 3
  • 1
0
votes
1 answer

When using tf.data.TFRecordDataset as the input pipeline, how to have sess.run() or eval() invoked more than once in the same iteration round?

With tensorflow, I've made a dataset = tf.data.TFRecordDataset(filename) and iterator = dataset.make_one_shot_iterator(). Then in each round iterator.get_next() would give out a mini-batch of data as input. I am training a network with Dropout…
Woody. Wang
  • 135
  • 1
  • 9
0
votes
3 answers

Dynamic switching of dropout in Keras/Tensorflow

I am building a reinforcement learning algorithm in Tensorflow and I would like to be able to dynamically turn dropout off and then on within one single call to session.run(). Rationale: I need to (1) do a forward pass w/o dropout to calculate the…
nicolas
  • 125
  • 9
0
votes
1 answer

Why do we want to scale outputs when using dropout?

From the dropout paper: "The idea is to use a single neural net at test time without dropout. The weights of this network are scaled-down versions of the trained weights. If a unit is retained with probability p during training, the outgoing…
MichaelSB
  • 3,131
  • 3
  • 26
  • 40
0
votes
1 answer

Keras weight file load exception: loading 2 layers into a model with 0 layers

Exception happened when I add dropout to the input layer. The exception was mentioned in other threads as well related to another issues and most common suggested solution is to downgrade the Keras version. Is there a workaround for this…
Gökhan Yu
  • 25
  • 6
0
votes
1 answer

Dropout in R: randomly remove elements and replace them with NA

I want to implement dropout i.e. randomly replace elements with NAs and return the vector/list back with NA values. The wanted result is to reduce overfitting so there may be better convenience functions for this. Approach 1 b<-rnorm(100);…
hhh
  • 50,788
  • 62
  • 179
  • 282
1 2 3
…
14 15