2

Does anyone had the experience with color matching and frame-to-frame tracking of it video when the exposure settings and white balance are constantly changing?

I'm working on color tracking app that uses iPad 2 frontal camera to capture video. I need to detect colored objects (of predefined color we took earlier) on each frame. My problem is that the camera software are like to adjust WB and Exposure each frame. So if we remember one color at frame N, on N+10 frame the WB will be different and this can lead to big difference in color.

For calculating color distance I'm using LAB color space and CIE76 formula: enter image description here

Yes, i know there is much better CIEDE2000 distance function, but I'm working with ARM processor and I'm afraid this formula will be too heavy even for ARM NEON manually optimized assembly code that i use already.

CIE76 provides a good results in general, but in poor or very bright lighting scenes the camera either generate too much noise or over-saturate the image so the colors becomes too far from their original. In addition to simple thresholding using color distance i implemented per-component thresholding of LAB pixel values based on standard deviation of the calibrated color. This had also increased correctness of the detection, however, this isn't solving the main issue.

The camera itself provide frames in RGB color space, but the API doesn't provide functions to get white point or color temperature of the current frame. Currently i assume D50 illuminant to perform RGB -> LAB conversion.

And this is the my main doubt. My idea is to compute the white point of the given RGB image, and then convert it to XYZ color space and then convert XYZ to LAB using calculated white point. Is it possible?

From Wikipedia: White Point

Expressing color as tristimulus coordinates in the LMS color space, one can "translate" the object's color according to the von Kries transform simply by scaling the LMS coordinates by the ratio of the maximum of the tristimulus values at both white points. This provides a simple, but rough estimate.

http://en.wikipedia.org/wiki/White_point

Are this going to work? Or there is a better way to calculate white point (even roughly)? By the way, i came out for Retinex algorithm, which demonstrate good color enhancement in shadows, does anyone used it? What it's pros and cons?

nbanic
  • 1,270
  • 1
  • 8
  • 11
BloodAxe
  • 833
  • 9
  • 10
  • You are suggesting the use of OpenCV but you are not mentioning any of its ready functions. Combining back projection (cvCalcBackProject) with CAMSHIFT tracking (cvCamShift) might be enough for your problem, also take a look into the demo `camshaftdemo.c`. There are too many considerations in your question but no actual approach has been tried. – mmgp Feb 10 '13 at 02:34
  • For color tracking, I'd look to the GPU instead of the CPU so that you can realize much faster performance. I've done a little experimentation with lamination-independent color tracking in the ColorObjectTracking example of my [GPUImage](https://github.com/BradLarson/GPUImage) framework, so you could examine the shader I was using there. It's based on this GPU Gems article: http://http.developer.nvidia.com/GPUGems3/gpugems3_ch26.html. Would working with the native YUV camera colorspace help in your normalization here? – Brad Larson Feb 10 '13 at 18:13
  • Hi Brad! I had used your GPUImage library in past and i have to say it's great and allows to do computations on GPU very easy. I'm considering both CPU and GPU optimizations of the final algorithm, but currently i'm looking for a robust solution of my needs. Thanks for sharing GPUGems link, but unfortunately the mathematical model of color difference in this article is totally wrong. The RGB color model they use is'n linear at all. But they use euclidian distance (L2 norm) to find the matching color areas. It's wrong. – BloodAxe Feb 11 '13 at 10:42
  • The LAB color space gives the best linear approximation of color and the CIE76 formula i mentioned in the question gives fairly good results. But i need to be able to detect the same color in different lighting conditions (different white balance). Like on picture below: http://photo.net/photodb/photo?photo_id=9470859 So my goal is to take the color of banana from the first image as reference color, and the find the banana on other images. So i'm looking for white balance correction algorithm where the white balance is actually unknown (have to estimate too). – BloodAxe Feb 11 '13 at 10:48

0 Answers0