0

Let's say I have a vector y_true, and a vector y_pred. I calculate the cross entropy between the two vectors, but for the gradient backpropagation, I don't know how to calculate the derivative of the cross entropy.

How to calculate the derivative of the cross entropy between two vectors? And is it possible to use this loss function with different activation functions?

I would like to solve the MNIST with a MLP, for that I made my own network without using a machine learning framework. My code for the calculation of the cross entropy is like this:

Cross_entropy() {};
    virtual double Compute(Eigen::MatrixXd y_true, Eigen::MatrixXd y_pred){
        double res;
        for (int i(0);i<y_true.cols();i++){
            res+= y_true(0,i)*log(y_pred(0,i));
        }
        return -res;
    }
    virtual Eigen::MatrixXd  Compute_prime(Eigen::MatrixXd y_true, Eigen::MatrixXd y_pred){
        Eigen::MatrixXd res = ??? ;
        return res;
    }
Nicolas Brugie
  • 35
  • 6
  • Please add a **language** tag. – desertnaut Apr 22 '21 at 11:40
  • You should figure this out on paper for the simplest network you can think of (1 input, 1 layer, 1 node). Before you've got all the maths worked out it doesn't make sense to start coding. – Swier Apr 22 '21 at 11:44

0 Answers0