0

I have an optimization problem I wish to solve that has some special characteristics. I have been trying to figure out how to fit it into the mold that SciPy optimize wants, but have been having some trouble. Could someone familiar with the package help me understand how to get what I want out of it?

My optimization formula is

min(A) sum P(yi=1|A) s.t. A.T*A == I

Where A is a matrix.

So I make a function opt_funct for the minimization function, but how do I pass it the matrix? Do I need to optimize a vector and then reshape the vector into the matrix within the optimization function?

For the constraint, I can make a function that returns A.T*A - eye(d), but I need to check that this is all zeros. Should I also reshape it as a vector, and will the constraint section of optimize know that every part of that vector needs to be 0?

Andrew Latham
  • 5,982
  • 14
  • 47
  • 87
  • 2
    Could you explain your objective function a bit more? What is `P(y | A)`? – ali_m Aug 04 '14 at 10:10
  • That's actually a tricky question because the paper I'm using basically says "make it up" but let's say that it's something really simple, like P(yi=1|A) = (# of values in row i > 0)/(length of row i) – Andrew Latham Aug 04 '14 at 14:53
  • In other words, you want to minimize `(A > 0).sum(1) / float(A.shape[1])` over `A` under your constraint? I think what you're describing is an [integer programming](http://en.wikipedia.org/wiki/Integer_programming) problem. These kinds of problem are *NP*-hard, and scipy's `optimize` module doesn't really have anything suitable for solving them. – ali_m Aug 04 '14 at 15:32
  • No, sorry, that was just an example. Let's just say that P is some custom function that I can write myself but that doesn't change this problem into anything simpler than what I've written above. – Andrew Latham Aug 04 '14 at 15:38
  • 1
    no, it does not make it any simpler, but for the general case your problem may be arbitrarily hard, ie unsolvable. and even with a simple P it is already pretty hard indeed, considering you are trying to solve a nonlinear matrix function. considering that A.T*A==I implies a rotation matrix, it is probably wise to exploit the underlying geometry of this rotation, rather than solve the problem in terms of a nonlinear function of a bunch of matrix coefficients. no out of the box optimization function is going to be able to cope with that. – Eelco Hoogendoorn Aug 04 '14 at 17:23
  • Could you at least give us a link to the paper you're referring to? Are you by any chance trying to implement the FastICA algorithm? – ali_m Aug 04 '14 at 18:58
  • Paper Link: http://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/aaai10midr.pdf The equation I wrote in the question is simpler than that in the paper because I wanted to get at the root of what I was trying to accomplish; namely, optimizing around a matrix rather than a set of variables. – Andrew Latham Aug 04 '14 at 22:01

0 Answers0