ANDROID - color detection using openCV - how to?

Question

my goal is to display a threshed image using the HSV color space in a way that only yellow objects will be shown. i use this code (based on a code given by the openCV 2.3.1 android samples):

protected Bitmap processFrame(VideoCapture capture) {
    //capture.retrieve(mGray, Highgui.CV_CAP_ANDROID_GREY_FRAME);
    //Imgproc.cvtColor(mGray, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);

    capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
    Imgproc.cvtColor(mHSV, mRgba, Imgproc.COLOR_RGB2HSV, 4);
    //Core.inRange(mRgba, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mRgba);

    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

    if (Utils.matToBitmap(mRgba, bmp))
        return bmp;

    bmp.recycle();
    return null;
}

the base (Abstract)class contains the "run" method:

protected abstract Bitmap processFrame(VideoCapture capture);

public void run() {
...
bmp = processFrame(mCamera);
...
canvas.drawBitmap(bmp, (canvas.getWidth() - bmp.getWidth()) / 2, (canvas.getHeight() - bmp.getHeight()) / 2, null);
...
}

i get this distorted preview which i think i can understand (HSV format) but why is it repeating itself (i`v draw a green line to emphasize it) 4 time? and what is the black horizontal line? enter image description here what am i doing wrong?

one last thing, what is the logic behind:

Imgproc.cvtColor(mHSV, mRgba, Imgproc.COLOR_RGB2HSV, 4);

why is it COLOR_RGB2HSV? shouldnt it be COLOR_HSV2RGB?

Let's say i'v passed this problem, how can i make a gray level image with the yellow objects in their native color? i thought using the Core.inRange() method but when i do this i get black screen.

yes, i guess i look like a total jerk but i need to start from somewhere, don't i?

10x!

Update 1: i tried to do RGB->HSV->RGB this way:

 @Override
protected Bitmap processFrame(VideoCapture capture) {
    //capture.retrieve(mGray, Highgui.CV_CAP_ANDROID_GREY_FRAME);
    //Imgproc.cvtColor(mGray, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);

    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGB);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_RGB2HSV,0);
    //Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_BGR2RGB, 4);
    //Core.inRange(mRgba, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mRgba);
    Imgproc.cvtColor(mHSV,mRgba , Imgproc.COLOR_HSV2RGB,0);

    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

    if (Utils.matToBitmap(mRgba, bmp))
        return bmp;

    bmp.recycle();
    return null;
}

and i got: enter image description here

?

Update 2:

i finally understand that before setting a frame, it must be converted into RGBA space. so i now tried the threshold with the code as follow:

capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_RGB2HSV,0);
    Core.inRange(mHSV, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mHSVThreshed);
    Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_HSV2RGB, 0);
    Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_RGB2RGBA, 0);
    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

but now it gives me force shutdown... any ideas?

I know its some time ago and its not the correct way to ask - but have you done a tutorial for this? as i can not get the VideoCapture working i would be glad if you could provide your code - or parts of your code at github. Big Thanks — user1651460, Jan 03 '14 at 09:04
Hey there. I`m really sorry but it has been a while since i`v worked with openCV, Plus i don't have the code anymore. I advise you to look here: http://docs.opencv.org/doc/tutorials/highgui/video-write/video-write.html#videowritehighgui. Good luck, hard work pays eventually :) — gor, Jan 05 '14 at 07:54

gor · Answer 1 · 2012-02-27T20:38:01.833

15

friends. i give you the result of 1 month of hard work and help from friends across the ocean: enter image description here

Ethan was right. but the code needed some fixing.

the code:

    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_BGRA);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_BGR2HSV,3);
    Core.inRange(mHSV, new Scalar(0, 100, 30), new Scalar(5, 255, 255), mHSVThreshed);
    Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_GRAY2BGR, 0);
    Imgproc.cvtColor(mRgba, mRgba2, Imgproc.COLOR_BGR2RGBA, 0);
    Bitmap bmp = Bitmap.createBitmap(mRgba2.cols(), mRgba2.rows(), Bitmap.Config.ARGB_8888);


    if (Utils.matToBitmap(mRgba2, bmp))...

first, the mat is binary 0 or 255 so the transform to gray level is more "natural". second, the conversion from HSVto RGBis in fact HSV-BGR!!. and last thing is that the preview is expecting RGBA Bitmap.

thats it. hope other can benefit from this post. SHALOM!

edited Feb 27 '12 at 20:38

answered Feb 27 '12 at 19:58

gor

1,046
1
14
28

Kudos, this was a great help! – octopi Apr 18 '12 at 23:37
@gor could you please add the values of mRgba,mHSV,mHSVThreshed,mRgba2? – dvrm Dec 17 '15 at 07:52
not sure i understand the question + this was posted years back, and the code is no more accessible.. – gor Dec 18 '15 at 19:01
to many Imgproc.cvtColor... so for inRange we just need HSC as input and GRAY as output (RGB2HSV) and (GRAY2RGB) – user25 Mar 06 '17 at 20:55

Ethan Chen · Accepted Answer · 2012-02-27T16:48:27.667

5

I think mHSVThreshed is a binary mat

so maybe this line :

Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_HSV2RGB, 0);

should change to :

Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_GRAY2RGB, 0);

I spent a lot of time dealing with the "showing" problem too...

hope this help...

edited Feb 27 '12 at 16:48

answered Feb 27 '12 at 11:26

Ethan Chen

164
1
9

Hey i want to add a matrix on the highlighted portion in the image can you guide me? – Smit Patel May 09 '14 at 09:04
What kind of matrix? You mean draw something on it like AR? – Ethan Chen May 17 '14 at 03:29

score 2 · Answer 3 · answered Feb 25 '12 at 11:25

2

Well as far as I see it you fetch the image frame in RGBA and save it under the name "mHSV"

capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);

so you should there store it as mRgba

in the cvtColor you need to transform it to HSV via COLOR_RGBA2HSV. Assuming you have changed the names this would be:

Imgproc.cvtColor(mRgba, mRSV, Imgproc.COLOR_RGB2HSV, 0);

And I assume this repetition of the images comes from the "4" in you cvtColor function since your HSV picture will only have 3 channels. Put in a 0 there and it should be detected automatically...

I hope it helps...

answered Feb 25 '12 at 11:25

florianbaethge

2,520
3
22
29

Hey, exactly what I meant, but with better explanations :p – jlengrand Feb 25 '12 at 11:29
Did you try to debug to find out where exactly it throws an error? Like which commands seem to be completed without error and where does it quit? – florianbaethge Feb 26 '12 at 13:01
i did. apparently the app never get to the line : Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_BGR2RGBA, 0); – gor Feb 26 '12 at 19:02
before that the main thread do the "mCamera.grab()" and a couple of seconds after that it just crashes.. – gor Feb 26 '12 at 19:04
Not sure at all, but might it be that OpenCV doesnt like it when input and output pointers actually are the same object? Both mRgba... – florianbaethge Feb 26 '12 at 19:21

score 1 · Answer 4 · answered Feb 25 '12 at 11:19

why is it COLOR_RGB2HSV? shouldnt it be COLOR_HSV2RGB?

I would say that it should :). Looks like a problem on how opencv grabs the pixels. Check that both your input and output images have the same size and numberof channels.

This

capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);

looks also weird to me, Are you storing the RGBA image that you get into an image at HSV format? That would explain the problem. Try to do something like that : RGB(capture retrieve) => HSV(cvt color) => color detection => RGB (cvt color again)

i know it looks wired but when i try converting: capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA); Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_HSV2RGB_FULL, 4); it doesnt work... — gor, Feb 25 '12 at 11:34

ANDROID - color detection using openCV - how to?

4 Answers4

Linked