On the page http://mxnet.io/tutorials/python/mnist.html# the weight matrix dimension is incorrect

Question

Under Multilayer Perceptron subsection, the documentation says that weight matrix has a dimension of mk.However, I think it should be km because the output layer-Y has the dimension of n*k.

Is my understanding correct?

Answer 1

The correct matrix calculation formula for a FullyConnected layer is:

output = input x weight_transpose + bias

So in this formula, your input is n x m, your weight is k x m before being transposed, and your output is n x k. See the FullyConnected API doc here.

If you're just getting started, it may make sense to focus on the new Gluon API, which is far easier to learn and debug.