0

I'm building a trading bot and I'm trying to implement an optimiser to maximise alpha while adhering to certain constraints.

My variable is a vector containing the weights of the securities in the portfolio. I also have a vector that contains the respective alpha scores for each security. My objective function is -sum(weights*alphas) (to minimise negative alpha). The constraints that I have are:

  • Min and max weight of stock in the portfolio
  • Min trade (as a percentage of total portfolio value - there is a fixed cost to trade so I don't want to trade if it's just a couple of basis points change)
  • Max turnover (maximum total trade value as a percentage of total portfolio value)
  • The sum of the absolute weights must add up to 1
  • The sum of the weights must equal either 0 (for long/short) or 1 (for long only)

I have created a class below which implements this using scipy.optimize.minimize:

class Optimiser:

    def __init__(self, initial_portfolio, turnover, min_trade, max_wt, longshort=True):
        self.symbols = initial_portfolio.index.to_numpy()
        self.init_wt = initial_portfolio['weight'].to_numpy()
        self.alpha = initial_portfolio['alpha'].to_numpy()
        self.longshort = longshort
        self.turnover = turnover
        self.min_trade = self.init_wt.copy()
        self.set_min_trade(min_trade)
        self.max_wt = max_wt
        if self.longshort:
            self.wt_sum = 0
            self.abs_wt_sum = 1
        else:
            self.wt_sum = 1
            self.abs_wt_sum = 1

    def set_min_trade(self, min_trade):
        for i in range(len(self.init_wt)):
            if abs(self.init_wt[i]) > min_trade:
                self.min_trade[i] = 0.1

    def optimise(self):
        wt_bounds = self.get_stock_wt_bounds()
        constraints = self.get_constraints()
        result = minimize(
            fun=self.minimise_negative_alpha,
            x0=self.init_wt,
            bounds=wt_bounds,
            constraints=constraints,
            options={
                'disp': True,
            }
        )
        return result

    def minimise_negative_alpha(self, opt_wt):
        return -sum(opt_wt * self.alpha)

    def get_stock_wt_bounds(self):
        if self.longshort:
            return tuple((-self.max_wt, self.max_wt) for s in self.init_wt)
        else:
            return tuple((0, self.max_wt) for i in range(len(self.init_wt)))

    def get_constraints(self):
        min_trade = {'type': 'ineq', 'fun': self.min_trade_fn}
        turnover = {'type': 'ineq', 'fun': self.turnover_fn}
        wt_sum = {'type': 'eq', 'fun': self.wt_sum_fn}
        abs_wt_sum = {'type': 'eq', 'fun': self.abs_wt_sum_fn}
        return turnover, wt_sum, abs_wt_sum

    def min_trade_fn(self, opt_wt):
        return self.min_trade - abs(opt_wt - self.init_wt)

    def turnover_fn(self, opt_wt):
        return sum(abs(opt_wt - self.init_wt)) - self.turnover*2

    def wt_sum_fn(self, opt_wt):
        return sum(opt_wt)

    def abs_wt_sum_fn(self, opt_wt):
        return sum(abs(opt_wt)) - self.abs_wt_sum

As you can see I am not using the min_trade constraint and I'll touch upon this later in the question.

Here are two examples I'm passing into it (these examples only contain 4 stocks and in the proper implementation I am looking to pass arrays of 50-100 securities):

a)

def run_optimisation():
    initial_portfolio = pd.DataFrame({
        'symbol': ['AAPL', 'MSFT', 'GOOGL', 'TSLA'],
        'weight': [-0.3, -0.2, 0.45, 0.05],
        'alpha': [-0.2, -0.3, 0.25, 0],
    }).set_index('symbol')

    opt = Optimiser(initial_portfolio, turnover=0.3, min_trade=0.1, max_wt=0.4)
    result = opt.optimise()

b)

def run_optimisation():
    initial_portfolio = pd.DataFrame({
        'symbol': ['AAPL', 'MSFT', 'GOOGL', 'TSLA'],
        'weight': [-0.25, -0.25, 0.25, 0.25],
        'alpha': [-0.2, -0.3, 0.25, 0],
    }).set_index('symbol')

    opt = Optimiser(initial_portfolio, turnover=0.3, min_trade=0.1, max_wt=0.4)
    result = opt.optimise()

The result that I am getting from a) for this long-short example is: [-0.1, -0.4, 0.25, 0.25] which is obviously not optimal [-0.1, -0.4, 0.4, 0.1].

I am getting this message:

Optimization terminated successfully.    (Exit mode 0)
            Current function value: -0.20249999999999585
            Iterations: 7
            Function evaluations: 42
            Gradient evaluations: 7

It's saying it successfully found the minimum... It's like it's trying to max out the turnover constraint. Is this because the initial weights do not adhere to the constraints? If so, how can I modify that as ideally I would like to pass it the current weights of the portfolio as x0.

In b) I am getting the optimal solution [-0.1, -0.4, 0.4, 0.1] but I am getting a False for result.success.

I am also getting this message:

Positive directional derivative for linesearch    (Exit mode 8)
            Current function value: -0.23999999999776675
            Iterations: 8
            Function evaluations: 34
            Gradient evaluations: 4

I think this message may mean that it is unable to increase/decrease the objective function much with changes and thus it does not know if it is at the minimum, please correct me if I am wrong. I have tried messing around with the ftol setting to no avail although I am not too sure how to set it optimally.

Is there a way to modify this optimiser so that a) it achieves the optimal solution and produces the correct status accordingly and b) can take initial weights that do not conform to the constraints? The hope for the future is to also include sector and industry constraints so that I cannot be over-invested in certain areas.

Also, as a side question (albeit not as important as I would just like to get it working to begin with): How might I implement the min trade constraint? I would like it so that either the stock is not traded at all or it has a trade value over this amount or trading away it's full value (to zero weight if it's less than the min_trade weight in the portfolio).

As you can see this is a very long question but I would really appreciate any help, guidance or answers you could provide for this problem! Please ask for any clarification or extra information as this has taken me a long time to cobble together and I am probably not explaining something well or missing something. Thank you!

OllieHooper
  • 405
  • 3
  • 9
  • 1
    Make sure you understand at least some of the core principles of those solvers before jumping in. Those solvers are mostly general NLP-solvers, assuming twice differentiability everywhere (using abs is not twice diff!) and providing local-optima only (convexity!). I never did quant-finance stuff, but i would expect, that most of these problems are convex and should be handled by convexity-assuming solvers. E.g. using [cvxpy](https://www.cvxpy.org/) as framework, providing some solvers and automatic reformulation of some things like `abs` or `norm(x, 1)`. – sascha Apr 12 '20 at 13:10
  • 1
    Summary: look at convex-optimization and cvxpy's concept of DCP. Does it look like it fits? Use it -> automatic reformulation of some non-smooth stuff, no need for numerical-differentiation (hidden in your code) and proof of convexity = proof of local-optima = global-optima if DCP is able to formulate the problem. **Limitation:** min-trade looks like disjunctive-like constraint -> branch-and-bound needed (see LP vs. MIP; or MIQP, MISOCP, MICP) and without commercial-solvers, cvxpy is somewhat limited in supported solvers (there are not many).If cvxpy/dcp is not enough for your use-case:(MI)NLP – sascha Apr 12 '20 at 13:22
  • 1
    And in (MI)NLP world, there are ipopt, bonmin, couenne, shot (all part of CoinOR). But those solvers are also NLP-based = assuming twice-diff [(see this)](https://coin-or.github.io/Ipopt/) (meaning: you need to linearize your abs-based constraints; e.g. by helper-constraints), much harder to setup (probably only through pyomo with python), but much much more advanced as those in scipy. Things get a bit more nice when using julia instead of python. But MINLP + open-source/free + user-friendly formulation is full of hurdles. – sascha Apr 12 '20 at 13:24
  • Thank you so much Sascha, this was the help I definitely needed. I think I was expecting this problem to be a bit easier than it is. I'm looking forward to the challenge! – OllieHooper Apr 12 '20 at 14:16

1 Answers1

0

Following on from Sascha's comments above I wanted to post the correct implementation of this problem in cvxpy:

import cvxpy as cv


class Optimiser:

    def __init__(self, initial_portfolio, turnover, max_wt, longshort=True):
        self.symbols = initial_portfolio.index.to_numpy()
        self.init_wt = initial_portfolio['weight'].to_numpy()
        self.opt_wt = cv.Variable(self.init_wt.shape)
        self.alpha = initial_portfolio['alpha'].to_numpy()
        self.longshort = longshort
        self.turnover = turnover
        self.max_wt = max_wt
        if self.longshort:
            self.min_wt = -self.max_wt
            self.net_exposure = 0
            self.gross_exposure = 1
        else:
            self.min_wt = 0
            self.net_exposure = 1
            self.gross_exposure = 1

    def optimise(self):
        constraints = self.get_constraints()
        optimisation = cv.Problem(cv.Maximize(cv.sum(self.opt_wt*self.alpha)), constraints)
        optimisation.solve()
        if optimisation.status == 'optimal':
            print('Optimal solution found')
        else:
            print('Optimal solution not found')
        return optimisation.solution.primal_vars

    def get_constraints(self):
        min_wt = self.opt_wt >= self.min_wt
        max_wt = self.opt_wt <= self.max_wt
        turnover = cv.sum(cv.abs(self.opt_wt-self.init_wt)) <= self.turnover*2
        net_exposure = cv.sum(self.opt_wt) == self.net_exposure
        gross_exposure = cv.sum(cv.abs(self.opt_wt)) <= self.gross_exposure
        return [min_wt, max_wt, turnover, net_exposure, gross_exposure]

Many thanks to Sascha for the help and guidance.

OllieHooper
  • 405
  • 3
  • 9