0

I am totally new to NN and want to classify the almost 6000 images that belong to different games (gathered by IR). I used the steps introduced in the following link, but I get the same training accuracy in each round. some info about NN architecture: 2 convloutional, activation, and pooling layers. Activation type: relu, Number of filters in first and second layers are 30 and 70 respectively. 2 fully connected layer with 500 and 2 hidden layers respectively. http://firsttimeprogrammer.blogspot.de/2016/07/image-recognition-in-r-using.html

Najme Rastegar
  • 21
  • 2
  • 5
  • When you use mx.set.seed(100) it makes the process deterministic. – IRTFM Jun 25 '17 at 15:11
  • Please share your code – Qiang Kou Jun 26 '17 at 17:08
  • What do you mean by "I get the same training accuracy in each round"? Are you training with multiple epochs? If you reset the network every time, you will get the same overall accuracy. If you continue to train the model, the first few epochs should give you better and better results. – Guy Jul 04 '17 at 16:15

1 Answers1

1

I had a similar problem, but for regression. After trying several things (different optimizers, varying layers and nodes, learning rates, iterations etc), I found that the way initial values are given helps a lot. For instance I used a -random initializer with variance of 0.2 (initializer = mx.init.normal(0.2)).

I came upon this value from this blog. I recommend you read it. [EDIT]An excerpt from the same,

Weight initialization. Worry about the random initialization of the weights at the start of learning. If you are lazy, it is usually enough to do something like 0.02 * randn(num_params). A value at this scale tends to work surprisingly well over many different problems. Of course, smaller (or larger) values are also worth trying. If it doesn’t work well (say your neural network architecture is unusual and/or very deep), then you should initialize each weight matrix with the init_scale / sqrt(layer_width) * randn. In this case init_scale should be set to 0.1 or 1, or something like that. Random initialization is super important for deep and recurrent nets. If you don’t get it right, then it’ll look like the network doesn’t learn anything at all. But we know that neural networks learn once the conditions are set. Fun story: researchers believed, for many years, that SGD cannot train deep neural networks from random initializations. Every time they would try it, it wouldn’t work. Embarrassingly, they did not succeed because they used the “small random weights” for the initialization, which works great for shallow nets but simply doesn’t work for deep nets at all. When the nets are deep, the many weight matrices all multiply each other, so the effect of a suboptimal scale is amplified. But if your net is shallow, you can afford to be less careful with the random initialization, since SGD will just find a way to fix it. You’re now informed. Worry and care about your initialization. Try many different kinds of initialization. This effort will pay off. If the net doesn’t work at all (i.e., never “gets off the ground”), keep applying pressure to the random initialization. It’s the right thing to do.

http://yyue.blogspot.in/2015/01/a-brief-overview-of-deep-learning.html

AntonySamuelB
  • 143
  • 7