0

When I was training a CNN to classify images of distorted digits varying from 0 to 9, the accuracy of training set and test set improved obviously.

Epoch[0] Batch [100] Train-multi-accuracy_0=0.296000
...
Epoch[0] Batch [500] Train-multi-accuracy_0=0.881900

In Epoch[1] and Epoch[2] the accuracy oscillate slightly between 0.85 and 0.95, however,

Epoch[3] Batch [300] Train-multi-accuracy_0=0.926400
Epoch[3] Batch [400] Train-multi-accuracy_0=0.105300
Epoch[3] Batch [500] Train-multi-accuracy_0=0.098200

Since then, the accuracy was around 0.1 which meant the network only gave random prediction. I repeated the training several times, this case occurred every time. What's wrong with it? Is the adapted learning rate strategy the reason?

model = mx.model.FeedForward(...,
                             optimizer = 'adam',
                             num_epoch = 50,
                             wd = 0.00001,
                             ...,
                             )
Lolith
  • 33
  • 2
  • It's possible you've started to overfit, or hit a point where you need to reduce the learning rate or the model diverges. Are you using a learning rate schedule or is this with fixed learning rate? Can you monitor loss rather than accuracy? – Ben Allison Aug 22 '17 at 19:52

1 Answers1

1

What exactly is the model you're training? If you're using the mnist dataset, usually a simple 2-layer MLP trained with sgd with give you pretty high accuracy.

eric-haibin-lin
  • 377
  • 2
  • 9
  • My dataset is not MNIST, it's a synthetic library of distorted characters. What I am concerned is the reason of the abrupt drop. I need some suggestion to analysis this problem. – Lolith Jun 03 '17 at 02:50