CVXPY error when trying to solve a convex minimization / binary programming problem

Question

I'm trying to solve a problem that I submitted to QSE, https://quant.stackexchange.com/questions/65680/find-k-of-n-assets-that-minimize-the-correlation-matrix/, but I'm running into an issue using the cvxpy lib. Namely, what I believe to be a convex binary programming problem, cvxpy is saying "does not follow DCP rules".

The problem I'm trying to solve is: "from a given 10 risky assets, find the 5 least correlated". My current methodology to solve this is

1. Grab each asset's returns
2. Find the correlation matrix, C, for these returns
3. Formulate the convex optimization problem as
   • x is a binary vector
   • the sum of the entries of x is 5
   • C' = transpose(C * x) * x is a matrix where the ith row and ith column of C is "zero'd out" when the ith entry of x is 0 (and not zero'd out otherwise). Note: I wonder if this is where my issue arises. This was the best way that I could come up with to remove entries from the correlation matrix C that correspond to the "rejected" assets.
   • Finally, I want to minimize the sum of squares of C'. This will give me something like a portfolio of the 5 least correlated assets.

The following is the non-working code that I have so far.

import pandas as pd
import pandas_datareader as web
import datetime as dt
import cvxpy as cvx

stocks = ['SHW', 'GOOG', 'AMZN', 'WMT', 'XOM', 'JNJ', 'UPS', 'AMT', 'AAPL', 'NEE']
start = dt.datetime(2015, 1, 1)
end = dt.datetime(2020, 1, 1)
d = web.DataReader(stocks, 'yahoo', start, end)['Adj Close']
corr = d.corr().to_numpy()

x = cvx.Variable(len(stocks), boolean=True)
cost = cvx.sum_squares((corr @ x).T @ x)
prob = cvx.Problem(cvx.Minimize(cost), [cvx.sum(x) == 5])
prob.solve(solver='ECOS_BB')

And the error it produces

DCPError: Problem does not follow DCP rules. Specifically:
The objective is not DCP. Its following subexpressions are not:

# The corr array

I've also tried to reformulate this in a number of ways that have't worked including

• Using a cvxp variable matrix X that has cvxpy attributes boolean and symmetric. If it's symmetric and each row sums to 5, then I have a matrix for which I can do element-wise multiplication to find C'. This doesn't work because a variable is only allowed one attribute (strange cvxpy limitation).
• Using a cvxp variable binary matrix X and a cvxp variable symmetric matrix Y and including a constraint that X == Y (to get around the 2-attribute limitation). I can't remember why this didn't work.
• Using a cvxp variable binary matrix X and constraints that the sum of every ith row is 5 and that the ith row is equal to the jth column. I had trouble with this because testing X[i] == X[:i] produces a boolean array that I didn't know how to reduce with cvxpy.
• I've also tried using a non-binary cvxpy variable vector x and attempting to constrain the ith value of x to be 0 or 1 but the constraint x[i] == 0 || x[i] == 1 wasn't valid because of ||--I also couldn't find a cvxpy logical-or condition.

So I've tried to reformulate this in a few different ways but I keep running into issues with each strategy. I'm wondering if someone can help me

1. Determine if this actually is a problem that cvxpy can solve. If it's not, how can I change it to solve essentially what I want? If it is something that cvxpy can solve
2. What can I do with my current code to correct its issues?

Thanks for your time.

Answer 1

I think C' is not a matrix but a scalar:

transpose(C * x) * x = (Cx)'x=x'C'x=x'Cx

(I use ' for transpose here) So sum of squares of this would not make much sense?

Could your problem be just the cardinality-constrained portfolio problem?

Or streamlined:

  min x'Cx
  sum(x) = k
  x ∈ {0,1}

The rows and columns of C corresponding to the selected assets in x is the submatrix you are looking for.