2

i try to describe the preconditions first

i have a number of images/matrices that can be imagined to be layers in an image manipulation program. these layers be will be added to form the final output. each layer has a factor from 0 to 1

x_1 * M_1 + x_2 * M_2 + ... = S

the M_i matrices are fix and does not change. The goal is to determine the x_i factors that best (not necessarily the best) approximates a given S

The 2-dimensionality of the matrices is irrelevant for solving the problem. they can be rearranged to form a linear equation system in matrix form.

each matrix M_i is a column of M and together they form an equation

M · x = S

M can be very large (several 10.000 rows, and 100 columns)

do you have suggestions on methods preferably based on libraries to solve this? I know there are many libraries that can solve least square problems but i could not find one that supports constraints. at first fulfilling the >= 0 condition would do it for me

Thanks in advance for your help

phimuemue
  • 34,669
  • 9
  • 84
  • 115
vlad_tepesch
  • 6,681
  • 1
  • 38
  • 80
  • You did not specify the type of constraints that you have. A least square problem with affine constraints is a quadratic programming problem. Quadratic constraints -> QCQP. You might want to consider searching along this line. – lightalchemist Feb 10 '14 at 10:12
  • @lightalchemist i specified the constraints: "each layer has a factor from 0 to 1", "at first fulfilling the >= 0 condition would do it for me" – vlad_tepesch Feb 10 '14 at 10:19

2 Answers2

2

mlpack appears to have Lagrange multipliers, see the docs. These allow you to solve equations with constraints.

doctorlove
  • 18,872
  • 2
  • 46
  • 62
0

you can use NLopt library for optimization and construct your objective function as a computation of squered innovations (probably using COBYLA algorithm).

You can use also QuantLib's General Linear Leaast Squeres class for this, usage is as simple as:

LinearRegression *lr = new LinearRegression( x, y, 1.0);
cout << lr->coefficients();

QuantLib LinearRegression

4pie0
  • 29,204
  • 9
  • 82
  • 118
  • i not had the time to look into nlopt yet but i took a short look to QuantLib and i did not understood your example. what does the 1.0 mean? i could not find a class `LinearRegression` on the class list i estimate you mean GeneralLinearLeastSquares but its not very well documented. what does 1.0 mean and that is a vContainer – vlad_tepesch Feb 10 '14 at 11:13
  • I have used LinearRegression class before, it looks like it is not documented in details in Doxygen docs, however it is still there as we can find in the General Linear Leaast Squeres notes: Inherited by LinearLeastSquaresRegression< ArgumentType >, and LinearRegression. – 4pie0 Feb 10 '14 at 11:23
  • i took a look onto quantlib and linearregression. i think it has nothing common with my problem. it simply gets a few x,y coordinates and calculates polynom coefficents. or did i miss something? How should i apply it on my problem. besides this, the library s really huge. it took over an hour to build and produces static libs of size>600MB. – vlad_tepesch Feb 11 '14 at 14:47
  • the problem you described is regression. Packages which I mentioned offer meanings to solve regression equation – 4pie0 Feb 11 '14 at 15:24
  • ok.. then i do not understand how to apply it to my problem. lets assume "A*b = C" A is an 40000 x 500 matrix and b a 1 x 500 matrix and C a 500 x 1 matrix. i have A and C and the constraints that all elements of b have to be >= 0. how can i determine b with help of QuantLib? – vlad_tepesch Feb 12 '14 at 10:05
  • A should be 40000x500, B 500x1, C 40000x1. So your problem is to solve just linear equation system, not to estimate parameters. right? – 4pie0 Feb 12 '14 at 10:59
  • sorry, C of course 40000x1 and b is a column matrix. yes its an linear equation system as i wrote in my question. but iam looking for the best solution with all elements of b >=0. – vlad_tepesch Feb 12 '14 at 11:44
  • you have mentioned also least squares method, but this is not least squares problem, you need just solve a linear equation system. – 4pie0 Feb 12 '14 at 12:18