How to render OpenCV Mat using OpenGL ES 2.0 on Android NDK?

Question

I already have a cv::Mat in Android NDK

cv::Mat grayMat;
grayMat.create(480, 720, CV_8UC1);

and a TextureView in JAVA.

public class MainActivity extends Activity implements TextureView.SurfaceTextureListener{
   private TextureView mTextureView;

   @Override
   protected void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      mTextureView = new TextureView(this);
      mTextureView.setSurfaceTextureListener(this);
   }
   ....
}

Why: I'm doing some image processing using OpenCV in NDK. Now, I'm using the method that save the result to .jpg and show on ImageView. But next step I'll use camera preview to do some processing. I don't want to save each frames to see the results.

Goal: Show the cv::Mat on Android

Question: How to show the cv::Mat using OpenGL ES 2.0 in NDK??

score 0 · Answer 1 · answered Sep 27 '17 at 09:26

If you want to display the Mat in Android, you can translate the Mat to pixels, then pass the pixels to Android. In Android, you can translate the pixels to bitmap to display.

here is the code to get the pixels in Mat.

int* getPixel(Mat src){
    int w = src.cols;
    int h = src.rows;
    int* result = new int[w*h];

    int channel = src.channels();
    if(channel == 1){
        for (int i = 0; i < w; i++) {
            for (int j = 0; j < h; j++) {
                int value = (int)(*(src.data + src.step[0] * j + src.step[1] * i));
                result[j*w+i] = rgb(value, value, value);
            }
        }
    }else if(channel == 3 || channel == 4){
        for (int i = 0; i < w; i++){
            for (int j = 0; j < h; j++){
                int r = (int)(*(src.data + src.step[0] * j + src.step[1] * i + src.elemSize1() * 2));
                int g = (int)(*(src.data + src.step[0] * j + src.step[1] * i + src.elemSize1()));
                int b = (int)(*(src.data + src.step[0] * j + src.step[1] * i));
                result[j*w+i] = rgb(r, g, b);
            }
        }
    }
    return result;
}

int rgb(int red, int green, int blue) {
    return (0xFF << 24) | (red << 16) | (green << 8) | blue;
}

get bitmap with pixels(also need width and height).

Bitmap bmpReturn = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565);
bmpReturn.setPixels(matPixels, 0, width, 0, 0, width, height);

I found that I can use EGL and SurfaceView to solve the problem this morning. And start to study EGL. Your method is not what I need, I don't want to use Bitmap. I need faster way and closer to real-time. But thank you to this suggestion. — Julia Ding, Sep 28 '17 at 04:01

How to render OpenCV Mat using OpenGL ES 2.0 on Android NDK?

1 Answers1