0

I am using scipy.optimize.minimize to minimize a function with l2 norm constraints and non-negative constraints on the computed parameters (some related links 1, 2, 3). More specifically, I have tried

con = ({'type': 'ineq', 'fun': lambda x: x}, {'type': 'eq', 'fun': lambda x: np.dot(x.T, x) - 1})

or

con = {'type': 'eq', 'fun': lambda x: np.dot(x.T, x) - 1}
bounds = [(0., None)] * n_features

with method SLSQP. The the accuracy of the algorithm is fine If the positive constraint is not used. However, when the positive constraint is used, most of the computed parameters x became zero and the algorithm returns low accuracy. To avoid this situation, I have tried different initializations, e.g., non-negative x0 with l2 norm equal to 1, x0 values close to zero, x0 values close to 1 but didin't help. Any suggestions to improve the functionality of the algorithm are highly appreciated.

EDIT: A running example can be found here.

Thoth
  • 993
  • 12
  • 36
  • (1) A bound is better than a constraint. (2) This is nonconvex. See if a different starting point helps. – Erwin Kalvelagen Jan 14 '21 at 00:44
  • @ErwinKalvelagen thanks for the interest. Please let me share my implementation. I have tried different initialization but without success. Is it possible to review my code? It is Logistic Regression. – Thoth Jan 14 '21 at 10:35
  • I also tried to use `trust-constr` but I couldn't make it. The error `ValueError: all the input array dimensions except for the concatenation axis must match exactly` is returned. The same error is discussed [here](https://github.com/scipy/scipy/issues/9146) but I couldn't understand it. It possible to cast some light on this too? Maybe another optimization method will work more efficiently. – Thoth Jan 14 '21 at 10:40
  • The constraints for SLSQP and trust-constr are specified in a different format. This is easy to miss in the documentation. You also may want to provide some working code that can be executed and demonstrates the problem.. – Erwin Kalvelagen Jan 15 '21 at 10:17
  • @ErwinKalvelagen here is a [link](https://github.com/thothxp/nnLR-scipy) of the complete code. – Thoth Jan 15 '21 at 15:39

0 Answers0