I’m trying to create a neural network using numpy to get a better understanding of it. I am trying to calculate dL/da in the hidden layer after the output layer.
I wrote a 2-4-4-2 nn and found the formula dL/da_hidden = dL/da_nextLayer * diff(activationFunction(z)) where z is the weighted sum from this hidden layer, L is loss and a is activation. I use 4 batches, so dL/da_hidden should be 4x4 if I think correctly. But since dL/da_nextLayer is 4x2 (2 output neurons,4 batches) and the other term is 4x4, i can only receive a 4x2 matrix for dL/da_hidden. Do I use the wrong formula or have a thinking mistake?
