3D matrix multiplication in opencv for RGB color mixing

Question

I'm trying to perform an RGB Color mixing operation in opencv. I have the image contained in an MxNx3 Mat. I would like to multiple this with a 3x3 matrix. In Matlab I do the following: *Flatten the image from MxNx3 to a MNx3 *multiply the MNx3 matrix by the 3x3 color mixing matrix *reshape back to a MxNx3

In Opencv I would like to do the following:

void RGBMixing::mixColors(Mat &imData, Mat &rgbMixData)
{
   float rgbmix[] = {1.4237, -0.12364, -0.30003, -0.65221, 2.1936, -0.54141, -0.38854, -0.47458, 1.8631};
   Mat rgbMixMat(3, 3, CV_32F, rgbmix);
   // Scale the coefficents
   multiply(rgbMixMat, 1, rgbMixMat, 256);
   Mat temp = imData.reshape(0, 1);
   temp = temp.t();
   multiply(temp, rgbMixMat, rgbMixData);
}

This compiles but generates an exception:

OpenCV Error: Sizes of input arguments do not match (The operation is neither 'a rray op array' (where arrays have the same size and the same number of channels) , nor 'array op scalar', nor 'scalar op array') in arithm_op, file C:/slave/WinI nstallerMegaPack/src/opencv/modules/core/src/arithm.cpp, line 1253 terminate called after throwing an instance of 'cv::Exception'
what(): C:/slave/WinInstallerMegaPack/src/opencv/modules/core/src/arithm.cpp: 1253: error: (-209) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'sca lar op array' in function arithm_op

This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information.

---

Update 1:

This is code that appears to work:

void RGBMixing::mixColors(Mat &imData, Mat&rgbMixData)
{
    Size tempSize;
    uint32_t channels;

    float rgbmix[] = {1.4237, -0.12364, -0.30003, -0.65221, 2.1936, -0.54141, -0.38854, -0.47458, 1.8631};
    Mat rgbMixMat(3, 3, CV_32F, rgbmix);
    Mat flatImage = imData.reshape(1, 3);
    tempSize = flatImage.size();
    channels = flatImage.channels();
    cout << "temp channels: " << channels << " Size: " << tempSize.width << " x " << tempSize.height << endl;
    Mat flatFloatImage;
    flatImage.convertTo(flatFloatImage, CV_32F);
    Mat mixedImage = flatFloatImage.t() * rgbMixMat;
    mixedImage = mixedImage.t();
    rgbMixData = mixedImage.reshape(3, 1944);
    channels = rgbMixData.channels();
    tempSize = rgbMixData.size();
    cout << "temp channels: " << channels << " Size: " << tempSize.width << " x " << tempSize.height << endl;
}

But the resulting image is distorted. If I skip the multiplication of the two matrices and just assign

mixedImage = flatFloatImage

The resulting image looks fine (just not color mixed). So I must be doing something wrong, but am getting close.

Answer 1

I see a couple of things here:

1. For scaling the coefficients, OpenCV supports multiplication by scalar so instead of multiply(rgbMixMat, 1, rgbMixMat, 256); you should do directly rgbMixMat = 256 * rgbMixMat;.

2. If that is all your code, you don't properly initialize or assign values to imData, so the line Mat temp = imData.reshape(0, 1); is probably going to crash.

3. Assuming that imData is a MxNx3 (3-channel Mat), you want to reshape that into a MNx3 (1-channel). According to the documentation, when you write Mat temp = imData.reshape(0, 1); you are saying that you want the number of channels to remain the same, and the row, should be 1. Instead, it should be:

   Mat myData = Mat::ones(100, 100, CV_32FC3); // 100x100x3 matrix

   Mat myDataReshaped = myData.reshape(1, myData.rows*myData.cols); // 10000x3 matrix

4. Why do you take the transpose temp = temp.t(); ?

5. When you write multiply(temp, rgbMixMat, mixData);, this is the per-element product. You want the matrix product, so you just have to do mixData = myDataReshaped * rgbMixMat; (and then reshape that).

Edit: It crashes if you don't use the transpose, because you do imData.reshape(1, 3); instead of imData.reshape(1, imData.rows);

Try

void RGBMixing::mixColors(Mat &imData, Mat&rgbMixData)
{
    Size tempSize;
    uint32_t channels;

    float rgbmix[] = {1.4237, -0.12364, -0.30003, -0.65221, 2.1936, -0.54141, -0.38854, -0.47458, 1.8631};
    Mat rgbMixMat(3, 3, CV_32F, rgbmix);

    Mat flatImage = imData.reshape(1, imData.rows*imData.cols);
    Mat flatFloatImage;
    flatImage.convertTo(flatFloatImage, CV_32F);

    Mat mixedImage = flatFloatImage * rgbMixMat;

    rgbMixData = mixedImage.reshape(3, imData.rows); 
}