CVXPY Quadratic Programming - ArpackNoConvergence error and AssertionError

Question

I am trying to use the Python package CVXPY to solve a quadratic problem but I keep getting errors. Could you help me?

Here is my code:

p = lasso.p # This is a number
Sigma_opt = lasso.Sigma_opt # This is a positive semi-definite matrix of shape (p,p)
rho_pair = lasso.rho_pair # Vector of shape (p,)
mu = lasso.mu # Positive scalar number.

def solve_2nd_problem(p, Sigma_opt, rho_pair, mu):
  beta = cp.Variable(p) # Variable to optimize
  obj = cp.Minimize(0.5 * cp.quad_form(beta, Sigma_opt) - rho_pair.T @ beta + mu * cp.norm1(beta)) # Objective to minimize
  constraints = [] # Constraints

  # Solve the optimization problem
  prob = cp.Problem(obj, constraints)
  prob.solve()

  return beta.value

beta_opt = solve_2nd_problem(p, Sigma_opt, rho_pair, mu)

I get ArpackNoConvergence: ARPACK error -1: No convergence (11281 iterations, 0/1 eigenvectors converged) [ARPACK error -14: ZNAUPD did not find any eigenvalues to sufficient accuracy.]

After reading what was discussed here, I tried to change Sigma_opt by cp.psd_wrap(Sigma_opt) and to set cp.settings.EIGVAL_TOL = 1e-06, but doing so I am now getting this error:

AssertionError                            Traceback (most recent call last)
<ipython-input-34-1fbde9ad6b58> in <cell line: 19>()
     17   return beta.value
     18 
---> 19 beta_opt = solve_2nd_problem(p, Sigma_opt, rho_pair, mu)

11 frames
/usr/local/lib/python3.9/dist-packages/cvxpy/reductions/complex2real/complex2real.py in canonicalize_expr(self, expr, real_args, imag_args, real2imag, leaf_map)
    179             return result
    180         else:
--> 181             assert all(v is None for v in imag_args)
    182             real_out = expr.copy(real_args)
    183             return real_out, None

AssertionError: 

I am not sure if I well understood what was happening here, but I now for a fact that my matrix Sigma_opt is PSD (as np.all(np.linalg.eigvals(Sigma_opt) > 0) and (Sigma_opt.T == Sigma_opt).all() both return True).

The matrix Sigma_opt is of shape (p = 946, p = 946) and rho_pair is of shape (p = 946,).

Do you know how I could fix this issue?

Thanks.

Answer 1

There is a problem caused by a numerical instability in the solver used by CVXPY, it uses the SCS solver, which may have problems with large or ill-conditioned problems, , lets try a different one like MOSEK or ECOS, which might be more reliable.

First install it

!pip install mosek
!pip install ecos

then we can modify the solver_2nd_problem here is the code:

def solve_2nd_problem(p, Sigma_opt, rho_pair, mu):
    beta = cp.Variable(p) # Variable to optimize
    obj = cp.Minimize(0.5 * cp.quad_form(beta, Sigma_opt) - rho_pair.T @ beta + mu * cp.norm1(beta)) # Objective to minimize
    constraints = [] # Constraints

    # Solve the optimization problem
    prob = cp.Problem(obj, constraints)
    prob.solve(solver=cp.MOSEK, verbose=True)  # Change this to cp.ECOS if you prefer ECOS solver

    return beta.value