Questions tagged [trainingloss]

9 questions
1
vote
0 answers

Addressing Saddle Points in Keras Model Training

My keras model seems to have to hit a saddle point in it's training. Of course this is just an assumption; I'm not really sure. In any case, the loss stops at .0025 and nothing I have tried has worked to reduce the loss any further. What I have…
junfanbl
  • 451
  • 3
  • 21
1
vote
0 answers

How to understand a periodicity in the training loss using a pre-trained model of PyTorch?

I'm using a pre-trained model from Pytorch ( Resnet 18,34,50) in order to classify images. During the training, a weird periodicity appears in the training as you can see in the image below. Did somebody already have a similar issue?In order to deal…
1
vote
1 answer

Implemented model network but neither training error nor val error decreasing

Since I'm novice to Pytorch, this question might be a very trivial one, but I'd like to ask for your help about how to solve this one. I've implemented one network from a paper and used all hyper parameters and all layers described in the paper. But…
laura
  • 11
  • 1
1
vote
0 answers

About model comparison in terms of training/validation loss

I'm comparing two models, and want to clarify the weird results. Model 1 achieves lower training loss than model 2, but get higher validation loss. Because over-fitting and under-fitting are determined by comparing training/validation loss of…
0
votes
0 answers

What insights can the training and validation losses curve provide for my sound separation model's performance?

I trained sound separation model with 10000samples and the same number of sample was used in validation with number of epochs =10 and 32 batch and 4 number of worker using Adam optimizer . I need someone explain to me the output training and…
0
votes
0 answers

Training U-Net with Negatives

How to train a U-Net with negative examples? I trained U-Net with pictures of hands and fingers. The ground truth data are binary masks with white pixels for the foreground object (finger/hand) and black pixels for the background object. Now I want…
0
votes
2 answers

What happens if optimal training loss is too high

I am training a Transformer. In many of my setups I obtain validation and training loss that look like this: Then, I understand that I should stop training at around epoch 1. But then the training loss is very high. Is this a problem? Does the…
0
votes
1 answer

training loss during LSTM training is higher than validation loss

I am training an LSTM to predict a time series. I have tried an encoder-decoder, without any dropout. I divided my data n 70% training and 30% validation. The total points in the training set and validation set are around 107 and 47 respectively.…
Nhqazi
  • 732
  • 3
  • 12
  • 30
0
votes
0 answers

Validation loss is constant and training loss decreasing

I have a model training and I got this plot. It is over audio (about 70K of around 5-10s) and no augmentation is being done. I have tried the following to avoid overfitting: Reduce complexity of the model by reducing number of GRU cells and hidden…
swe87
  • 129
  • 1
  • 3
  • 13