0

I have been going through the logistic regression source code for some time now. See https://github.com/knime/knime-core/blob/master/org.knime.base/src/org/knime/base/node/mine/regression/logistic/learner3/Learner.java

My understanding on Logistic Regression is

1) Inialise a weight vector.
2) for each of the instance (until convergence following steps proceed)
       find response using y = (1/exp(-W_transp*X))   
       find gradient error = y - comp_y         //target - computed y
       append d = d + error*x;    // d is a vector initialised to 0 aftr evry epoch
       weight = weight + learningrate * d  //after every epoch updated

I don't understand what actually the complex code in irlsRls method does. Two of the Array2DRowRealMatrix xTwx and xTyu are populated but the mechanism done is unclear and for what are these used is also not mentioned.

for (int k = 0; k < tcC - 1; k++) {
                for (int i = 0; i < rC + 1; i++) {
                    for (int ii = i; ii < rC + 1; ii++) {
                        int o = k * (rC + 1);
                        double v = xTwx.getEntry(o + i, o + ii);
                        double w = pi.getEntry(0, k) * (1 - pi.getEntry(0, k));
                        v += x.getEntry(0, i) * w * x.getEntry(0, ii);
                        xTwx.setEntry(o + i, o + ii, v);
                        xTwx.setEntry(o + ii, o + i, v);
                    }
                }
            }
            // fill the rest of xTwx (k != k')
            for (int k = 0; k < tcC - 1; k++) {
                for (int kk = k + 1; kk < tcC - 1; kk++) {
                    for (int i = 0; i < rC + 1; i++) {
                        for (int ii = i; ii < rC + 1; ii++) {
                            int o1 = k * (rC + 1);
                            int o2 = kk * (rC + 1);
                            double v = xTwx.getEntry(o1 + i, o2 + ii);
                            double w = -pi.getEntry(0, k) * pi.getEntry(0, kk);
                            v += x.getEntry(0, i) * w * x.getEntry(0, ii);
                            xTwx.setEntry(o1 + i, o2 + ii, v);
                            xTwx.setEntry(o1 + ii, o2 + i, v);
                            xTwx.setEntry(o2 + ii, o1 + i, v);
                            xTwx.setEntry(o2 + i, o1 + ii, v);
                        }
                    }
                }
            }

            int g = (int)row.getTarget();
            // fill matrix xTyu
            for (int k = 0; k < tcC - 1; k++) {
                for (int i = 0; i < rC + 1; i++) {
                    int o = k * (rC + 1);
                    double v = xTyu.getEntry(o + i, 0);
                    double y = k == g ? 1 : 0;
                    v += (y - pi.getEntry(0, k)) * x.getEntry(0, i);
                    xTyu.setEntry(o + i, 0, v);
                }
}

Can someone guide. Thanx in advance.

Devi
  • 109
  • 1
  • 1
  • 10

0 Answers0