Multiple Linear Regression in C#

Question

I want to make a multiple linear regression in C#. I am trying to achieve this with MathNet.Numerics, but I keep getting the error "Matrix dimensions must agree 2x3".

/*****************************************************
    * MathNet.Numerics Multiple linear regression test *
    ****************************************************/

var ydata = new double[] { 15, 20, 25, 55, 95 };
var xdata = new[] { new[] { 1.0, 4.0 }, new[] { 2.0, 5.0 }, new[] { 3.0, 6.0 } };

var x = DenseMatrix.OfColumns(2, 3, xdata);
var y = new DenseVector(ydata);

var p = x.QR().Solve(y); // error: matrix dimensions must agree 2x3
var a = p[0];
var b = p[1];

MessageBox.Show(a.ToString(), "Test");
MessageBox.Show(b.ToString(), "Test");

Hmmm...mismatch of 5 values in ydata with 6 (2x3) values in xdata? — BlackjacketMack, Apr 18 '14 at 00:04

score 2 · Accepted Answer · answered Apr 18 '14 at 02:43

The exception text is really bad in this case, we should fix this.

There are two concrete problems causing this to fail:

The system with 3 unknowns but only 2 equations/samples (2x3 matrix) is under-defined; Applying a regression to such a problem does not actually make any sense as there is an infinite number of exact solutions.
For these two samples/equations the right hand side y has actually 5 values (instead of 2, to match each sample)

Maybe it is easier to start with the new Fit class available in v3. For example, let's do an ordinary regression with the following samples:

a    b         y
1.0, 4.0  -->  15.0
2.0, 5.0  -->  20.0
3.0, 2.0  -->  10.0

With the problem to find u, v and w that minimize the error with these samples in

y ~= u + a*v + b*w

With the Fit class this would look like the following:

double[] p = Fit.MultiDim(
    new[] {new[] { 1.0, 4.0 }, new[] { 2.0, 5.0 }, new[] { 3.0, 2.0 }},
    new[] { 15.0, 20, 10 },
    intercept: true);
// p = [u,v,w] = [-1.25,1.25,3.75], hence y ~= -1.25 + 1.25*a + 3.75*b

With linear algebra and matrix decompositions the same could look like:

var A = DenseMatrix.OfArray(new[,] {
           { 1.0, 1.0, 4.0 },
           { 1.0, 2.0, 5.0 },
           { 1.0, 3.0, 2.0 }});
var b = new DenseVector(new[] { 15.0, 20, 10 });
var p2 = A.Solve(b);
// p = [-1.25,1.25,3.75]

Does that help?

Perfect. Thanks. There's a small typo: "A.QR().Solve(b);" right? It worked that way. — Christoph Bühler, Apr 18 '14 at 11:02
On v2 you need to explicitly choose the QR decomposition, yes. In v3 there is a new Matrix.Solve method that automatically selects one - in this case it would have chosen the faster LU decomposition since the matrix is square. — Christoph Rüegg, Apr 18 '14 at 11:13

Multiple Linear Regression in C#

1 Answers1