How can I get mxnet back end code for various functions?

Question

I am trying to understand how the internal flow goes in mxnet when we call forward . Is there any way to get source code of mxnet?

Answer 1

This really depends on what your symbolic graph looks like. I assume you use MXNet with Python (Python documentation). There you can choose to use the MXNet symbol library or the Gluon library.

Now, you were asking whether one can inspect the code, and, yes, you can find it on GitHub. The folder python contains the python interface and src contains all MXNet sources. What happens on forward is eventually defined by the MXNet execution engine, which tracks input/output dependencies of operators and neural network layers, allocate memory on the different devices (CPU, GPUs). There is a general architecture documentation for this.

I suppose you are interested in what each and every operation does, such as argmax (reduction), tanh (unary math operation) or convolution (complex neural network operation). This you can find in the operator folder of MXNet. This requires a whole documentation in itself and there is a special forum for MXNet specifics here, but I will give a short orientation:

• Each operation in a (symbolic) execution graph needs a defined forward and backward operation. It also needs to define its output shape, so that it can be chained with other operations. If that operator needs weights, it needs to define the amount of memory it requires, so MXNet can allocate it.
• Each operation requires several implementations for a) CPU b) GPU (CUDA) c) wrapper around cuDNN
• All unary math operations follow the same pattern, so they are all defined in a similar way in mshadow_op.h (e.g. relu).

This is all I can tell you based on your quite broad question.