after merging and extracting the BGR channel, compilation is successful,but there is a run-time error: ILLEGAL OPERATION

Question

I am trying to create 1 CV_8UC3 image from 3 different CV_8UC1 images that I already have, i.e I am trying to allocate the different single channel images that I already have into a single 1 Multi-Dimensional Image. Likely the below code worked flawlessly directly with 3-channel image but If am merging and extracting it comes up with a Runtime error. ILLEGAL OPERATION

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include<vector>
typedef cv::Vec3b Pixel;  //  correct


struct Operator {
    void operator ()(cv::Vec3b &pixel, const int * position) const 
    {    
            pixel[2]*=0.5; 
    }
}; 

int main(int argc, char** argv )
{
    cv::VideoCapture cap(0);
    if(!cap.isOpened())  
        return -1;
    cv::Mat frame1,frame2,for_each,cblue, cgreen, cred; 
    std::vector<cv::Mat> channels { cblue, cgreen, cred};
    for(;;)
    {
            cap >> frame1;    
            cvtColor(frame1, frame1, cv::COLOR_BGR2GRAY); 
            frame1.convertTo(frame2,CV_8U);

            frame2.copyTo(cblue);
            frame2.copyTo(cgreen);
            frame2.copyTo(cred);

            cv::merge(channels, for_each);

            double t1 = (double)cv::getTickCount();
            for_each.forEach<Pixel>(Operator());
            t1 = ((double)cv::getTickCount() - t1)/cv::getTickFrequency();
            std::cout<< "Parallel TEST time " << t1 << std::endl;

            cv::extractChannel (for_each, cblue, 0 );
            cv::imshow("cropped_BGR",frame1);
            cv::imshow("mod_BLUE",cblue);

           if (cv::waitKey(30) == 27) 
           {
                std::cout << "esc key is pressed by user" <<std::endl;
                 break; 
           }
    }
    return 0;

}

I am getting no idea from where this error is coming, any help will be really appreciated , TIA.

Answer 1

The Problem:

When you do this:

cv::Mat frame1,frame2,for_each,cblue, cgreen, cred; 
std::vector<cv::Mat> channels { cblue, cgreen, cred};

channels will have a shallow copy of the cv::Mat cblue, cgreen and cred. This means, that they both will have the same headers with a data pointer that will point to the same place.

Then you do:

frame2.copyTo(cblue);
frame2.copyTo(cgreen);
frame2.copyTo(cred);

which does a deep copy of frame2 to each of the cv::Mat. The documentation of copyTo says:

m – Destination matrix. If it does not have a proper size or type before the operation, it is reallocated.

This means, the pointer to data will change, however it won't change to the cv::Mat inside the vector, they will still point to nullptr but cblue, cgreen and cred will point to the other place.

I tested it with this code:

  cv::Mat frame(500, 500, CV_8UC3, cv::Scalar::all(111));
  cv::Mat frame1, frame2, cblue, cgreen, cred;
  std::vector<cv::Mat> channels{ cblue, cgreen, cred };
  // at this point all data members of mat will point to nullptr except frame
  cv::cvtColor(frame, frame, cv::COLOR_BGR2GRAY);
  frame.convertTo(frame2, CV_8U);

  frame2.copyTo(cblue);
  frame2.copyTo(cgreen);
  frame2.copyTo(cred);

  // at this point all point to another place except the ones inside the vector

Possible solutions:

1) Create references and not copies:

  cv::Mat frame1, frame2;
  std::vector<cv::Mat> channels(3);
  cv::Mat& cblue = channels[0], &cgreen=channels[1], &cred=channels[2];

---

2) Use the channels directly instead of using other variables

        frame2.copyTo(channels[0]);
        frame2.copyTo(channels[1]);
        frame2.copyTo(channels[2]);

---

3) Create the vector inside the loop

        frame2.copyTo(cblue);
        frame2.copyTo(cgreen);
        frame2.copyTo(cred);
        std::vector<cv::Mat> channels { cblue, cgreen, cred};
        cv::merge(channels, for_each);

---

4) Your code is equivalent to:

cvtColor(frame1, frame1, cv::COLOR_BGR2GRAY);
cvtColor(frame1, for_each, cv::COLOR_GRAY2BGR);

This will create a 3 channel image of the grey values, which is basically a copy of the grey mat in each channel...

---

5) One more thing:

frame1.convertTo(frame2,CV_8U);

This is not necessary, because it is already a CV_8U mat, because the previous instruction converted it to greyscale which is CV_8U and then you can even create a vector there without doing deep copy (it will deep copy to for_each).

    std::vector<cv::Mat> channels { frame1, frame1, frame1};
    cv::merge(channels, for_each);

---

And one additional thing, not related to the error:

        cv::extractChannel (for_each, cblue, 0 );
        cv::imshow("cropped_BGR",frame1);
        cv::imshow("mod_BLUE",cblue);

will display exactly the same image :) or should at least.

Answer 2

Indeed, one of the best replies. Thank you very much !!!

sol3 : it worked perfectly

double t1 = (double)cv::getTickCount();
    std::vector<cv::Mat> channels { cblue, cgreen, cred};
    cv::merge(channels, for_each);
    for_each.forEach<Pixel>(Operator());
    cv::extractChannel (for_each, cblue, 2 );
t1 = ((double)cv::getTickCount() - t1)/cv::getTickFrequency();
std::cout<< "Parallel TEST time " << t1 << std::endl;

sol1: however it is still giving me

error: 'class std::vector' has no member named 'forEach' channels.forEach(Operator()); ^~~~~~~ 16_vector_foreach_changedmaincode.cpp:54:31: error: expected primary-expression before '>' token channels.forEach(Operator());

my intention was to

    frame2.copyTo(cblue);
    frame2.copyTo(cgreen);
    frame2.copyTo(cred);

double t1 = (double)cv::getTickCount();
    //std::vector<cv::Mat> channels { cblue, cgreen, cred};
    std::vector<cv::Mat> channels(3);
    cv::Mat& cblue = channels[0], &cgreen=channels[1], &cred=channels[2];
    //cv::merge(channels, for_each);
    channels.forEach<Pixel>(Operator());
    cv::extractChannel (channels, cblue, 2 );
t1 = ((double)cv::getTickCount() - t1)/cv::getTickFrequency();
std::cout<< "Parallel TEST time " << t1 << std::endl;

do I have to use the merge operator here also ?? @api55