Stack Overflow
Stack Overflow

Questions tagged [learning-rate]

83 questions
0
votes
1 answer

Pytorch1.6 What is the actual learning rate during training?

I'd like to know the actual learning rate during training, here is my code. learning_rate = 0.001 optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate) scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[1, 2],…
wwxiaokucha
  • 25
  • 5
0
votes
1 answer

Darknet yolo v4 - how to get current learning rate?

When I train yolo v4 with Darknet, few times a second I get a list of variables. Is there an easy way to make Darknet binary also print learning rate? If everything fails I will make changes in…
Huxwell
  • 171
  • 2
  • 14
0
votes
1 answer

Nan losses using "Learning Rate Step Decay" Scheduler with Adam Optimizer in Keras?

I have this very deep model: def get_model2(mask_kind): decay = 0.0 inp_1 = keras.Input(shape=(64, 101, 1), name="RST_inputs") x = layers.Conv2D(256, kernel_size=(3, 3), kernel_regularizer=l2(1e-6), strides=(3, 3), padding="same")(inp_1) …
Ciccios_1518
  • 435
  • 1
  • 6
  • 13
0
votes
1 answer

Why models often benefit from reducing the learning rate during training

In Keras official documentation for ReduceLROnPlateau class (https://keras.io/api/callbacks/reduce_lr_on_plateau/) they mention that "Models often benefit from reducing the learning rate" Why is that so? It's counter-intuitive for me at least,…
21kc
  • 21
  • 6
0
votes
1 answer

Decaying the learning rate from the 100th epoch

Knowing that learning_rate = 0.0004 optimizer = torch.optim.Adam( model.parameters(), lr=learning_rate, betas=(0.5, 0.999) ) is there a way of decaying the learning rate from the 100th epoch? Is this a good practice: decayRate =…
Robert
  • 7
  • 3
0
votes
1 answer

Directly update the optimizer learning rate

I have a specific learning rate schedule in mind. It is based on the epoch but differs from the generally available ones I am aware of including StepLR. Is there something that would perform the equivalent…
WestCoastProjects
  • 58,982
  • 91
  • 316
  • 560
0
votes
1 answer

How can I converge loss to a lower value?(tensorflow)

I used tensorflow object detection API. Here is my environment. All images are from coco API Tensorflow version : 1.13.1 Tensorboard version : 1.13.1 Number of test images : 3000 Number of train images : 24000 Pre-trained model : SSD mobilenet v2…
YanAwithU
  • 3
  • 4
0
votes
1 answer

Tensorboard plot ReduceLROnPlateau

I keep failing to plot my learning rate in tensorboard because I am using the ReduceLROnPlateau as following: tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=results_path, histogram_freq=1) reduce_lr =…
Ollowain
  • 55
  • 6
0
votes
2 answers

How to change the learning rate in PyTorch (1.6)

I am using PyTorch and I want to change the learning rate after some epochs. However, the code that is provided on most documentations, which is: optimizer = torch.optim.Adam([ dict(params=model.parameters(), lr=learning_rate), ]) …
Timbus Calin
  • 13,809
  • 5
  • 41
  • 59
0
votes
1 answer

tf.compat.v1.train.exponential_decay: global step = 0

To find out how to implement both a ANN with exponential decay as well as a with a constant learning rate I looked it up here: https://www.tensorflow.org/api_docs/python/tf/compat/v1/train/exponential_decay I have some questions: ... global_step =…
Christopher J McCoy
  • 7
  • 7
0
votes
0 answers

Cant build a deep learning model as mentioned in the arxiv paper

I am trying to replicate this paper using the same dataset. But somehow they trained and achieved(80%) a lot better accuracy than I usually(65%) get. My data is imbalanced 300:500(filler words: non-filler words). The model tends to overfit in the…
saisriteja kuppa
  • 15
  • 4
0
votes
2 answers

GridSearchCV for learning rate

I'm trying to use GridSearchCV to find the best parameters in a CNN but when I try to find the best combination for learning rate and batch size the code doesn't work (it works if I use instead of learning rate, epochs). Any idea why it is not…
juanc409
  • 15
  • 1
  • 5
0
votes
1 answer

Learning rate curve going backwards for cnn

I'm working on recognizing the numbers 3 and 7 using MNIST data set. I'm using cnn_learner() function from fastai library. When I plotted the learning rate, the curve started going backwards after a certain value on X-axis. Can someone please…
MonkeyDLuffy
  • 508
  • 1
  • 5
  • 24
0
votes
1 answer

Monitor the Learning Rate of the InverseTimeDecay() - float() argument must be a string or a number, not 'InverseTimeDecay'

My goal is to have a look at the learning rate progression of the Adam optimizer, on which I apply InverseTimeDecay schedule. So I want to check if the learning rate actually decreases. Having checked this question on stack overflow, I made similar…
NikSp
  • 1,262
  • 2
  • 19
  • 42
0
votes
0 answers

Why is learning rate get constant result (not change)?

When I use object detection with Faster R-CNN I get this result in tensorboard: Why do I get a constant result in my learning rate? What does that mean? Shouldn't my learning rate change each iteration?
xMAHEQSA
  • 1
1 2 3 4
5
6