3

I'm trying to do a 2d histogram in Python on an HSV image, but even using numpy and opencv it isn't fast enough (I'm doing it with video actually, but consider each frame to be just an image).

I'm looking for the Hue value that is most saturated. Currently I have the following code, which works ok, but is just too slow.

hist, xbins, ybins = np.histogram2d(hsv_channels[0].ravel(), saturation_channel.ravel(), [180,256],[[0,180],[0,256]])

I'm hoping to do this with PyOpenCL instead, and push the computation to the GPU, but aside from the hello world programs in OpenCL. I've found some papers on doing them, but I'm unsure where to start.

How would I get started with this?

EDIT:

I've thought about this a little more. I think the steps on the GPU I want to do are roughly the following:

  1. Turn image into a 1d array (if 10x10, turn into 100 long array)
  2. Upload image to GPU
  3. split image into n slices for processing, where n is the number of parallel compute units. Or each can reference a specific range on this array.
  4. (Map) For each compute unit, allocate 180 'bins' that can contain 256 other bins each. The innermost content of each is just an integer for counting.
  5. For each hue (one of 180 bins), count how many of that hue there are for each saturation level (the other 256 bins). Do this for the sub-section of the array that can be counted on.
  6. Create a new empty set of bins.
  7. (Reduce) For all of these bin counts, then merge them together (adding values). I'm unsure if I need to wait until they are all done, or just each in sequence to merge in with the empty bins from above.
  8. (determine final answer) For the final set of bins, loop through them and find the maximum saturation value for that hue, and store this. Now find the hue with the maximum saturation. As the final answer, return this hue # and this maximum saturation #.

Still, I don't know enough about GPU things with PyOpenCL (or OpenCL overall) to do this correctly.

ivan_pozdeev
  • 33,874
  • 19
  • 107
  • 152
tibbon
  • 1,018
  • 2
  • 16
  • 30
  • You can start here: http://stackoverflow.com/questions/5863979/opencl-image-histogram – Robert Ioffe Mar 04 '16 at 17:07
  • 4
    Basic questions: what is the video resolution, and how fast do you need it to be - what frame rate are you targeting? Also, do you need a precise answer, or is an approximation with small and known error bound (say, 1%, 5%, etc, at most) acceptable? – Francesco Callari Mar 09 '16 at 15:41
  • Let's say it's a 1920x1080 image. An approximation within 5% of the right hue/saturation values are probably ok. – tibbon Mar 14 '16 at 20:26
  • 15-30fps is the range I'd like to run in, so somewhere around 30-60ms per frame max. – tibbon Mar 14 '16 at 20:27
  • Although it's not OpenCL related, have you considered resizing the frame and then calculating the histogram on a much smaller scale? If you're going for a real-time approximation, that would be my first try. Though OpenCL does sound interesting, unless you're working on GPU loaded data, you'll have to pay the cost of memory-to-GPU transfer for every frame. While it might not be significant, it's yet another thing that will impact your overall performance. – bconstanzo Mar 15 '16 at 15:26
  • That's a really good point, and i hadn't considered it. – tibbon Mar 18 '16 at 15:37
  • I've gotten it using the 2d Histogram on my Macbook Pro to run at around 20ms/frame when I downscale it by 10x. The results still seem good, so that's ok. I think when I put it on a lesser device however, that we're still seeing it around 200ms/frame, which is less than ideal. – tibbon Mar 18 '16 at 15:39

1 Answers1

6

If you take a different approach you can reduce the computation time to about 2% of the time taken by histogram2d. On a 3143x2095 image this method took about 5 ms while histogram2d wasted about 280 ms.

import cv2
import numpy as np
from numpy import unravel_index
import time

img = cv2.imread('ducks.jpg')

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

print hsv.shape
hue = hsv[:, :, 0]
sat = hsv[:, :, 1]

start = time.time()
max_index = unravel_index(sat.argmax(), sat.shape)
end = time.time()
print 'argmax time:', end - start
print sat[max_index]
print hue[max_index]

start = time.time()
hist, xbins, ybins = np.histogram2d(hue.ravel(), sat.ravel(), [180, 256], [[0, 180], [0, 256]])
end = time.time()
print 'histogram2d time:', end - start

Output:

(2095, 3143, 3)
argmax time: 0.00526285171509
255
39
histogram2d time: 0.288522958755

It's easy to handle the case of having more than one pixel with the maximum saturation value.

fireant
  • 14,080
  • 4
  • 39
  • 48
  • Wow, this method is *way* faster indeed. I think however, that it isn't returning results quite as accurately as I'd ideally have. I super appreciate what you've got here though, and I'm wondering if there's a good way to tweak it to do exactly what I need. – tibbon Mar 18 '16 at 15:36
  • Nothing is inherently wrong with this. You could upload a sample image and the desired result plus the code that is not getting what you want. I'm sure you could get more help that way. – fireant Mar 18 '16 at 16:53
  • So in the end, what you gave me is actually *perfect*! – tibbon Mar 22 '16 at 15:24
  • The missing piece was that I had to blur the hell out of the image first, and then it returned within 1% the saturation/hue that the 2d histogram was. The problem otherwise was that one stray pixel that was super saturated (even just a hot pixel from the sensor) would mess it up. Blurring it made all the difference. Now it's fast and returning solid values. I'd still love to figure out the OpenCL map/reduce some day for fun, but this is good. Check it out at https://github.com/tibbon/lights/blob/master/lights.py – tibbon Mar 22 '16 at 15:26