Meaning of Parameters in BackgroundSubtractorGMG Algorithm in OpenCV 3.0

Question

I am studying the GMG background subtraction algorithm as described in this paper. As OpenCV 3.0 also has an implementation of the GMG algorithm (as the additional package opencv_contrib), I try to study the two together. However, I am not quite sure about the meaning of the two parameters maxFeatures and quantizationLevels, as I want to map them to the descriptions in the paper.

Quoting the source code in OpenCV 3.0 (file modules\bgsegm\src\bgfg_gmg.cpp):

//! Total number of distinct colors to maintain in histogram.
int maxFeatures;

and

//! Number of discrete levels in each channel to be used in histograms.
int quantizationLevels;

And quoting the paper (Section II B) (with some mathematical symbol and variable names modified since LaTex is not supported here):

"... Of the T observed features, select the F_tot <= F_max most recently observed unique features; let I is a subset of {1, 2, ... T}, where |I| = F_tot, be the corresponding time index set. (If T > F_max, it is possible that F_tot, the number of distinct features observed, exceeds the limit F_max. In that case, we throw away the oldest observations so that F_tot <= F_max). Then, we calculate an average to generate the initial histogram: H(T) = (1/F_tot)∑f(r). This puts equal weight, 1/F_tot, in F_tot unique bins of the histogram."

From the above description, I was originally convinced that maxFeatures in OpenCV 3.0 refers to F_max in the paper, and quantizationLevels refers to F_tot. However, this does not sound right for two reasons: (1) The paper mentions "F_tot is the number of distinct features observed", and (2) the OpenCV source code does not pose any relationship between maxFeatures and quantizationLevels, while the paper clearly suggests that the former should be larger than or equal to the latter.

So, what are the meaning of maxFeatures and quantizationLevels? And is quantizationLevels a parameter introduced by OpenCV for the calculation of histogram?

score 2 · Answer 1 · answered Sep 30 '15 at 10:38

After further studying the source code in OpenCV, I believe that maxFeatures refer to F_max in the paper, while quantizationLevels is actually the number of bins in the histogram. The reason is as follows:

In the function insertFeature(), which contains the following code:

static bool insertFeature(unsigned int color, float weight, unsigned int* colors, float* weights, int& nfeatures, int maxFeatures)
{
    int idx = -1;
    for (int i = 0; i < nfeatures; ++i) {
        if (color == colors[i]) {
            // feature in histogram
            weight += weights[i];
            idx = i;
            break;
        }
    }
    if (idx >= 0) {                          // case 1
        // move feature to beginning of list
        ::memmove(colors + 1, colors, idx * sizeof(unsigned int));
        ::memmove(weights + 1, weights, idx * sizeof(float));
        colors[0] = color;
        weights[0] = weight;
    }
    else if (nfeatures == maxFeatures) {     // case 2
        // discard oldest feature
        ::memmove(colors + 1, colors, (nfeatures - 1) * sizeof(unsigned int));
        ::memmove(weights + 1, weights, (nfeatures - 1) * sizeof(float));
        colors[0] = color;
        weights[0] = weight;
    }
    else {                                   // case 3
        colors[nfeatures] = color;
        weights[nfeatures] = weight;
        ++nfeatures;
        return true;
    }
    return false;
}

Case 1: When the color matches one item in the array colors[], the corresponding array element is obtained.

Case 2: When the color does not match any item in the array colors[] and maxFeatures is reached (nFeatures stores the number of items stored in the array), then the oldest feature is removed for the new color.

Case 3: When the color does not match any item in the array colors[] and maxFeatures is not reached yet, add the color to a new item of the array, and nFeatures is incremented by 1.

Hence maxFeatures should correspond to F_max (the maximum number of features) in the paper.

In addition, in the function apply():

static unsigned int apply(const void* src_, int x, int cn, double minVal, double maxVal, int quantizationLevels)
{
    const T* src = static_cast<const T*>(src_);
    src += x * cn;

    unsigned int res = 0;
    for (int i = 0, shift = 0; i < cn; ++i, ++src, shift += 8)
        res |= static_cast<int>((*src - minVal) * quantizationLevels / (maxVal - minVal)) << shift;

    return res;
}

This function maps the pixel's color intensity pointed to by the pointer src_ to a bin according to the values of maxVal, minVal and quantizationLevels, such that if quantizationLevels = q, the result of the code:

static_cast<int>((*src - minVal) * quantizationLevels / (maxVal - minVal))

must be an integer in the range [0, q-1]. However, there can be cn channels (for example, in RGB, cn = 3, hence the shift operation and the possible number of bins (denote it as b) is therefore quantizationLevels to the power of cn. So if b > F_max, we have to discard the (b - F_max) oldest features.

Hence, in OpenCV, if we set maxFeatures to be >= quantizationLevels ^ cn, we never have to discard the oldest features, because we allow more than enough bins, or more than enough different unique features.

Meaning of Parameters in BackgroundSubtractorGMG Algorithm in OpenCV 3.0

1 Answers1