The specific mathematical implementation of DeepLearning4J

Question

I am trying to determine the exact mathematics used to train feed forward networks used for classification in deeplearning4j with stochastic gradient descent. I have tried stepping through the code but am getting lost in the forest.

Is this documented anywhere?

Answer 1

You can find it in the preOutput and backpropGradient methods in the various layers implementations:

Here's the basic implementation used in DenseLayers

And the more complex convolution one used in Convolutions

The implementation might be swapped out when using CuDNN or MKL however.