0

I'm trying to implement the activation function tanh on my CNN, but it doesn't work, the result is always "NaN". So i created a simple application where i have a randomized matrix and try to apply the tanh(x) function thus to understand where's the problem?

Here's my implementation : 

    Eigen::MatrixXd A = Eigen::MatrixXd::Random(10,1000);
    Eigen::MatrixXd result, deriv;
    result = A.array().tanh();
    deriv = 1.0 - result*result;

and the only result to this is this error : 

no match for ‘operator-’ (operand types are ‘double’ and ‘const Eigen::Product<Eigen::Matrix<double, -1, -1>, Eigen::Matrix<double, -1, -1>, 0>’)
     deriv = (1.0 - result*result );
              ~~~~^~~~~~~~~~~~~~~

Could you please help me ?

Sabrina Tesla
  • 13
  • 1
  • 10
  • 1
    So your code is trying to assign a matrix to a scalar value? That's not going to work. – john Aug 06 '18 at 09:04
  • Can you give me a solution to this please? The input is a Matrix of weights so i'm obliged to use Matrix – Sabrina Tesla Aug 06 '18 at 09:07
  • I don't have a solution, a matrix and a scalar are two different things. I'm not sure what you expect to happen. – john Aug 06 '18 at 09:08
  • There's broadly speaking nothing "wrong" with calling `tanh` on a matrix, but the result is going to be a matrix as well. (Formally matrices belong in the domain of linear algebra and `tanh` doesn't, but you're probably abusing a matrix class to store an image) – MSalters Aug 06 '18 at 09:10
  • thank you @MSalters it was a useful comment. I did that, and then there's another error : no match for ‘operator-’ (operand types are ‘double’ and ‘const Eigen::Product, Eigen::Matrix, 0>’) deriv = (1.0 - result*result ); ~~~~^~~~~~~~~~~~~~~ – Sabrina Tesla Aug 06 '18 at 09:20
  • you can't do sum or substration between scalar and matrix.... (hint : deriv should be a matrix too) – Jérémy Blain Aug 06 '18 at 09:26
  • Yes, i thought to make an eigen matrix of ones ... but didn't find how ? I find how to do it with Eigen::MatrixXd::Zeros and Eigen::Matrix::Identity but not with ones ... – Sabrina Tesla Aug 06 '18 at 09:28
  • 1
    @SabrinaTesla: You might want to read up on `auto`; you can let expression type determine variable type. Still, that won't allow you to do the mathematically impossible. – MSalters Aug 06 '18 at 09:39

2 Answers2

0

The product result*result does not have the right dimensions for a matrix multiplication. We can use result*result.transpose() instead (unless a coefficient-wise multiplication is intended, in which case one could use result.array()*result.array()).

To subtract the values of the resulting matrix from a matrix full of ones, the .array() method can be used:

deriv = 1. - (result*result.transpose()).array();
RHertel
  • 23,412
  • 5
  • 38
  • 64
0

I used openCV to create a matrix of ones like this : 

cv::Mat sum;
Eigen::MatrixXd SUM, Acv;
cv::eigen2cv(A,Acv)
sum=Mat::ones(Acv.rows,Acv.cols, CV_32FC1);
cv::cv2eigen(sum,SUM);

so :

deriv = SUM - result*result;

and now, here's another problem :( 

/usr/include/eigen3/Eigen/src/Core/CwiseBinaryOp.h :110 : Eigen::CwiseBinaryOp<BinaryOp, Lhs, Rhs>::CwiseBinaryOp(const Lhs&, const Rhs&, const BinaryOp&) [with BinaryOp = Eigen::internal::scalar_difference_op<double, double>; LhsType = const Eigen::Matrix<double, -1, -1>; RhsType = const Eigen::Product<Eigen::Matrix<double, -1, -1>, Eigen::Matrix<double, -1, -1>, 0>; Eigen::CwiseBinaryOp<BinaryOp, Lhs, Rhs>::Lhs = Eigen::Matrix<double, -1, -1>; Eigen::CwiseBinaryOp<BinaryOp, Lhs, Rhs>::Rhs = Eigen::Product<Eigen::Matrix<double, -1, -1>, Eigen::Matrix<double, -1, -1>, 0>]:  l'assertion « aLhs.rows() == aRhs.rows() && aLhs.cols() == aRhs.cols() » a échoué.
Sabrina Tesla
  • 13
  • 1
  • 10
  • 1
    The problem that you are facing now is due to the wrong dimensions in the matrix product `result*result`. – RHertel Aug 06 '18 at 09:46
  • Thank you very much for your help @RHertel, i did that and it works on my test.cpp, but now, i implemented this to my CNN's code, and there's a little problem :( – Sabrina Tesla Aug 06 '18 at 09:53
  • /usr/include/eigen3/Eigen/src/Core/CwiseBinaryOp.h :110 : Eigen::CwiseBinaryOp::CwiseBinaryOp(const Lhs&, const Rhs&, const BinaryOp&) [with BinaryOp = Eigen::internal::scalar_product_op; LhsType = const Eigen::Matrix; RhsType = const Eigen::Matrix; Eigen::CwiseBinaryOp::Lhs = Eigen::Matrix; Eigen::CwiseBinaryOp::Rhs = Eigen::Matrix]: l'assertion « aLhs.rows() == aRhs.rows() && aLhs.cols() == aRhs.cols() » a échoué. – Sabrina Tesla Aug 06 '18 at 09:56