Algorithm design: Image quantization for most prominent colors

Question

So I'm working on a way to extract dominant colors as perceived by humans from an image.

As an example, here's a photo: https://500px.com/photo/63897015/looking-out-for-her-kittens-by-daniel-paulsson

Most humans would think the 'dominant' color is that piercing azure of the eyes. Using standard quantization, however, that blue disappears completely when you drop below 16 colors or so. The eyes only take up 0.2% of the canvas, so going for the average doesn't work at all.

Project Details: I'm building a Rails app that will accept an uploaded photo or a specific color, and will return a bunch of other photos with similar dominant colors. The tool will be used by designers to find stock photography that matches their pre-existing color scheme. There are other cool ideas I have, too, if I can get the technical bit sorted out.

Current Research: I've spent the last 24 hours reading all about this stuff. Of all the services I've tried, TinEye is the only one that does it properly, but they're closed source. I can't provide more than 1 link, but you can google 'TinEye Color' to find it.

Tools Used: I'm using ImageMagick to do the image conversion and histogram generation.

Desired Result: When given that photo, I'd like to create a palette of 5-6 colors, with that saturated azure being one of them.

Current Method: The way I'm doing it now is I reduce it to 32 or 64 colors, and look for saturation/hue outliers. If the standard deviation is low and the outlier is significantly far away, I'll add it to the palette. Then I'll further reduce the photo to 4 colors and combine them for a 5-6 color palette.

My Question: My problem is with ImageMagick's quantization algorithm. I don't want to take the average of the colors, I want to group them by like colors and take the median, favoring saturated colors. When I reduce the cat picture to 32 colors, the eyes get turned into a desaturated grey.

So I'm asking you guys if you know of any algorithm or color principle that will let me find colors that stand out from an image, without blurring them together. I want a color that is actually in the image, not the blurred mean of 4-5 colors combined.

In a more general sense, TinEye has done a perfect job, and I'd like to figure out how they've done it.

Answer 1

You say, that the 'dominant' color was 'that piercing azure of the eyes'.

You also say, that you want as a result 'that saturated azure being one of' the 5-6 colors in your extracted palette.

You finally want 'a color that is actually in the image, not the blurred mean of 4-5 colors combined.'.

To start tackling this task, I've first tried to get to the exact color definition of 'that piercing azure of the eyes'.

But here I'm stuck already: that piercing azure is already a mix of at least 20 different shades of blue! See here, a picture that is a cropped cat eye, at 1000% zoom level:

So which one exactly do you want? So what if the best match for the blue you imagine in the end is a 'blurred mean' of several colors combined, and which does not appear in the original image even once?!

---

Update

Here is my first shot at it...

1. Use -posterize 8 to go to fewer colors

convert 2048.jpg -posterize 8 posterized-8.png

Here is the cat's eye when posterized and zoomed at 1000%:

2. Create a textual 'histogram' describing the 12 most frequently used colors after posterization

convert              \
   posterized-8.png  \
  -format %c         \
  -colorspace lab    \
  -colors 12         \
   histogram:info:-  | sort -n -r

  850708: (172,171,171) #ACABAB srgb(172,171,171)
  370610: (219,219,226) #DBDBE2 srgb(219,219,226)
  262870: (218,201,183) #DAC9B7 srgb(218,201,183)
  162588: (182,182,219) #B6B6DB srgb(182,182,219)
  161739: (182,219,219) #B6DBDB srgb(182,219,219)
  115671: ( 92, 87, 87) #5C5757 srgb(92,87,87)
  102337: (146,109,109) #926D6D srgb(146,109,109)
   86318: ( 67, 46, 46) #432E2E srgb(67,46,46)
   82882: ( 22, 20, 21) #161415 srgb(22,20,21)
   66221: (109,139,154) #6D8B9A srgb(109,139,154)
   58403: (146,146,109) #92926D srgb(146,146,109)
   38949: ( 97,109,146) #616D92 srgb(97,109,146)

3. Use the 12 most frequent colors to create a patch strip showing the palette:

convert      \
   -size 100x100 \
    xc:"srgb(172,171,171)" \
    xc:"srgb(219,219,226)" \
    xc:"srgb(134,119,120)" \
    xc:"srgb(182,182,219)" \
    xc:"srgb(182,219,219)" \
    xc:"srgb(92,87,87)" \
    xc:"srgb(146,109,109)" \
    xc:"srgb(67,46,46)" \
    xc:"srgb(22,20,21)" \
    xc:"srgb(109,139,154)" \
    xc:"srgb(146,146,109)" \
    xc:"srgb(97,109,146)" \
   +append \
    palette.png

This is how the palette looks like (it misses the colors from the very bright spot in the eye):

Answer 2

Here is an initial thought... I, or someone else, may develop it further. It was prompted by your statement that you want to favour the saturated colours.

First, remove all blacks and whites from the photo since they are unsaturated. Then convert to HSL colorspace and extract the Saturation channel. Contrast stretch the saturation to full range and then use this as a mask when applying ImageMagick's colour quantisation algorithms.

convert cat.png -fuzz 20% -fill black \
    -opaque white                     \
    -opaque black                     \
    -colorspace HSL -channel S -separate -contrast-stretch 0.1% out.png