Why does Armadillo fail to learn a Gaussian Mixture Model and complain about 'no existing means' despite random_subset seeding?

Question

QUESTION SUMMARY:

I have a [5 x 72580] matrix. I am trying to fit a Gaussian Mixture Model (GMM) to this data using the gmm_diag.learn() method with random_subset as the initial seeding mode. Why does Armadillo display "gmm_diag::learn(): no existing means" and fail to learn the model?

PROBLEM DETAILS:

I am working on a machine learning algorithm, the aim of which is to identify a writer from their handwriting. I am using supervised learning to train our model with a GMM.

All the training data is read from XML files. After calculating the features, their values are stored into a linked list. After this, the number of elements in the list is counted and is used to initialize an Armadillo mat(rix) variable at runtime as shown below:

int totFeatureVectors = CountPointClusterElements(TRAINING_CLUSTER_LIST_INDEX);
printf("\n%d elements added to list\n",totFeatureVectors);

mat F = mat(NUM_POINT_BASED_FEATURES, totFeatureVectors, fill::zeros);

Here TRAINING_CLUSTER_LIST_INDEX and NUM_POINT_BASED_FEATURES are a couple of configurable, project level constants; for my program NUM_POINT_BASED_FEATURES = 5 and totFeatureVectors = 72580. So the variable F is a [5 x 72580] dimensional matrix of double values. After initialization, I am reading the feature values from the linked list into F as follows:

int rowInd=0, colInd=0;
PointClusterElement *iterator = allClusterPointsList;
while(iterator!=NULL)
{
    F(rowInd,colInd)=iterator->pointSample.speed;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.dirn.cosComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.dirn.sinComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.curv.cosComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.curv.sinComponent;
    rowInd += 1;
    if(rowInd==NUM_POINT_BASED_FEATURES)
    {
        rowInd=0;
        colInd += 1;
    }
    iterator=iterator->nextClusterElement;
}

The assignment of feature values to locations in F is being made in a column major manner i.e. each column of F represents a feature vector post assignment. I am even writing the values of F into a text file to verify that all the feature values have been properly set and yes, it is happening without any problems

FILE *fp = fopen(PROGRAM_DATA_OUTPUT_PATH,"w");
if(fp!=NULL)
{
    int r,c;
    for(c=0; c<totFeatureVectors; c++)
    {
        for(r=0; r<NUM_POINT_BASED_FEATURES; r++)
        {
            fprintf(fp,"%lf\t",F(r,c));
        }
        fprintf(fp,"\n");
    }
}
fclose(fp);

So far, so good. But after this, when I declare a gmm_diag variable and try to fit a GMM to F using its learn() method, the program displays a warning "gmm_diag::learn(): no existing means" and quits, thus failing to learn the GMM (here the VARIANCE_FLOORING_FACTOR = 0.001)

gmm_diag writerModel;
bool result = writerModel.learn(F, 20, maha_dist, random_subset, 100, 100, VARIANCE_FLOORING_FACTOR, true);
writerModel.dcovs.print("covariances:\n");
writerModel.hefts.print("weights:\n");
writerModel.means.print("means:\n");
if(result==true)
{
    printf("\nModel learnt");
}
else if(result==false)
{
    printf("\nModel not learnt");
}

I opened up the learn() method on my IDE and from what I could make out, this error (warning) message is displayed only when the initial seeding mode is keep_existing. The source file I referred to is at /usr/include/armadillo_bits/gmm_diag_meat.hpp

My question is - why would this happen even when my seeding is done using the random_subset mode? How exactly should I proceed to get my model to learn? Not sure what I am missing here... The documentation and code samples provided at http://arma.sourceforge.net/docs.html#gmm_diag were not too helpful (the short program here works even without initializing the means of the GMM). The code is given below

int main(int argc, char** argv) {

int totFeatureVectors = CountPointClusterElements(TRAINING_CLUSTER_LIST_INDEX);
printf("\n%d elements added to list\n",totFeatureVectors);

mat F = mat(NUM_POINT_BASED_FEATURES, totFeatureVectors, fill::zeros);
int rowInd=0, colInd=0;
PointClusterElement *iterator = allClusterPointsList;
while(iterator!=NULL)
{
    F(rowInd,colInd)=iterator->pointSample.speed;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.dirn.cosComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.dirn.sinComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.curv.cosComponent;
    rowInd += 1;
    F(rowInd,colInd)=iterator->pointSample.curv.sinComponent;
    rowInd += 1;
    if(rowInd==NUM_POINT_BASED_FEATURES)
    {
        rowInd=0;
        colInd += 1;
    }
    iterator=iterator->nextClusterElement;
}

FILE *fp = fopen(PROGRAM_DATA_OUTPUT_PATH,"w");
if(fp!=NULL)
{
    int r,c;
    for(c=0; c<totFeatureVectors; c++)
    {
        for(r=0; r<NUM_POINT_BASED_FEATURES; r++)
        {
            fprintf(fp,"%lf\t",F(r,c));
        }
        fprintf(fp,"\n");
    }
}
fclose(fp);

gmm_diag writerModel;
bool result = writerModel.learn(F, 20, maha_dist, random_subset, 100, 100, VARIANCE_FLOORING_FACTOR, true);
writerModel.dcovs.print("covariances:\n");
writerModel.hefts.print("weights:\n");
writerModel.means.print("means:\n");
if(result==true)
{
    printf("\nModel learnt");
}
else if(result==false)
{
    printf("\nModel not learnt");
}

getchar();
return 0;}

TECHNICAL DETAILS:

The program is being run on a Ubuntu 14.04 OS using a Netbeans 8.0.2 IDE. The project is a C/C++ application

Any help would be most appreciated! Thanks in advance ~ Sid

score 3 · Answer 1 · answered Sep 12 '15 at 17:07

You need to try the simplest possible case first, in order to narrow down the location of the bug. Your code is certainly not simple, and it's also not reproducible (nobody except you has all the functions).

The following simple code works, which suggests that the bug is somewhere else in your code.

I suspect your code is overwriting memory somewhere, leading to data and/or code corruption. The bug is probably an incorrect pointer, or incorrectly used pointer.

#include <fstream>
#include <armadillo>

using namespace std;
using namespace arma;

int main(int argc, char** argv) {

  mat F(5,72580, fill::randu);

  gmm_diag model;

  bool result = model.learn(F, 20, maha_dist, random_subset, 100, 100, 0.001, true);

  model.hefts.print("hefts:");
  model.means.print("means:");
  model.dcovs.print("dcovs:");

  return 0;
  }

Output from above code:

gmm_diag::learn(): generating initial means
gmm_diag::learn(): k-means: iteration:    1   delta: 0.343504
gmm_diag::learn(): k-means: iteration:    2   delta: 0.0528804
...
gmm_diag::learn(): k-means: iteration:  100   delta: 3.02294e-06
gmm_diag::learn(): generating initial covariances
gmm_diag::learn(): EM: iteration:    1   avg_log_p: -0.624274
gmm_diag::learn(): EM: iteration:    2   avg_log_p: -0.586567
...
gmm_diag::learn(): EM: iteration:  100   avg_log_p: -0.472182
hefts:
   0.0915   0.0335   0.0308   ...
means:
   0.4677   0.1230   0.8582   ...
   ...
dcovs:
   0.0474   0.0059   0.0080   ...
   ...

Hello @mtall Thanks for your reply. I have already tried out the code snippet you suggested actually, even before posting the question. True that it does work. However if there was to be a memory corruption or pointer related issue somewhere, then wouldn't the IDE be notifying me by throwing some errors / segmentation faults etc.? I am pretty sure there isn't a problem upto the point where the linked list has been loaded with values. Is there any way I could share one of the XML files here along with the complete code? — sid, Sep 14 '15 at 05:44
If you have a random memory corruption, the results are undefined. You cannot rely on a segfault. The program may or may not crash, depending on where and how extensive the memory corruption is. — mtall, Sep 14 '15 at 08:04
Well now that you mention this, I must add that the original code is entirely written in C and not C++; i.e the complete creation and handling of the linked list uses C alone. My very limited experience with C++ prevents me from knowing for sure if letting pure C code run inside a C++ project is a good idea. It was only because of the fact that I was unable to find any machine learning libraries made for C that I was compelled to try Armadillo out. Anyway I shall try running valgrind on my code (sans Armadillo) to check if memory corruption is indeed happening - and then get back to you @mtall — sid, Sep 14 '15 at 11:06
I don't think mixing C and C++ has anything to do with this. You can certainly use C inside C++, as C++ is essentially a superset of C. — mtall, Sep 18 '15 at 02:04

johnwbyrd · Answer 2 · 2015-12-05T21:34:12.407

0

That branch can only be taken in the Armadillo code when seed_mode is equal to keep_existing, and the means matrix is empty. Why not simply do source-level debugging using your IDE and see at what point it's changing out from under you?

edited Dec 05 '15 at 21:34

answered Sep 25 '15 at 02:58

johnwbyrd

3,432
2
29
25

Why does Armadillo fail to learn a Gaussian Mixture Model and complain about 'no existing means' despite random_subset seeding?

2 Answers2