Fastest way to perform thousands of comparisons

Question

I'm working on image processing tool which translates RGB colors to human readable keywords. I need to:

1. Initialize/Declare 1000 static known elements set
2. Perform 10 comparisons between a dynamic element and the whole 1000 elements (total 10x1000 comparisons)
3. Repeat steps 1 & 2 thousands of images.

Each element is a Struct like this one:

struct KnownColor{
    cv::Scalar rgb;
    std::string name;
    std::vector<std::string> strings;

    KnownColor ( cv::Scalar rgb, std::string name, std::vector<std::string> strings) :
        rgb(rgb),
        name(name),
        strings(strings) {};
};

Currently I'm using a static std::vector<> to store them. To init this vector I'm currently using a function called once, and I push my 1000 elements into the vector. Every struct is initialised using the KnownColor constructor:

static std::vector<CustomStruct> Vector;    

void vectorInit() //I call this only once per image:
{
    ...
    Vector.push_back(KnownColor(Scalar(188,198,204),"Metallic Silver",std::vector<std::string>{"gray"}));
    Vector.push_back(KnownColor(Scalar(152,175,199),"Blue Gray",std::vector<std::string>{"gray","blue"}));
    ... (1000 push_back in total)
}

The comparison function, is a custom Euclidean distance between the Scalars which returns the closest "knownColor" and its human readable keywords:

double min = std::numeric_limits<double>::max();
int index=0;
for(int i=0; i<KnownColors.size(); i++)
{
    double tmp = scalarLABDistance(rgb,KnownColors[i].rgb);
    if(tmp < distanceThreshold )
    {
        min = tmp;
        break;
    }
    if(tmp < min )
    {
        min = tmp;
        index = i;
    }
}
return KnownColors[index].strings;

Sadly for now, the program is called from PHP and needs to be executed once per image (client request).

I wonder if a static init would be better (init is faster? iteration is faster?), or if there is a better way to compare something to a collection of static elements without using std::vector.

Answer 1

What I have done in similar situations is write a program to build a C file for me with the data structure that I need already sorted out. Usually I write this in Perl or Python. The data might be generated or it might be read from data files.

You can build a Boost array at compile time which would not require time for initialization. However, writing a 1000 elements into a vector should be super-fast on any desktop type CPU.

Searching your 1000 elements is more of a problem.

You said you are looking for the euclidean distance. What game developers have used to solve this is a technique called Binary Space Partition. That may help you. The way it works, very roughly, is that a 2D space is divided into 4 squares. A 3D space is divided into 8 cubes. Then each of those spaces is further divided, down to the smallest useful unit. This gives a quick way to search a tree. It gets complicated when searching for areas that don't fit into one partition though. You might decide to search multiple trees or put duplicates of objects in all trees it touches. For circles or spheres you might fall back to a perfect distance calculation or you might just subdivide it into squares/cubes until the smallest unit and call it close enough. There are a lot of examples and literature.

And, I am almost certain that you CAN hash this because a point in a 2D BSP is a string of 2-bit location values which is just an integer of various sizes depending on your BSP resolution.

It looks like another good option for this might be a kd-tree. That's a "k" dimensional tree. I haven't used one but from what I just read about them they look rather useful for this.