scipy optimization model returns all nan solutions

Question

I have the code below where I'm trying to calculate the max sharpe ratio solution to the efficient frontier. I'm trying to minimize the volatility and maximize the sharpe ratio (by minimizing the negative of the sharpe ratio). however my optimization model solution is returning all nan for the weights. I see the std value is nan as well. the covariance dataframe and the mean_return don't contain any nan values. I've included some sample data below. my actual prices_df has 253 records and 587 columns but all similar values to the examples below. for a dataset with a much smaller number of columns (14) the code below works fine. does anyone see what the issue might be?

sample data:

print(prices_df[prices_df.columns[:5]].head(n=5))

Equity(772 [CY])  Equity(128 [ALL])  \
2014-12-31 00:00:00+00:00            13.733             68.948   
2015-01-02 00:00:00+00:00            13.906             68.820   
2015-01-05 00:00:00+00:00            13.675             67.456   
2015-01-06 00:00:00+00:00            13.165             67.407   
2015-01-07 00:00:00+00:00            13.512             68.516   

                           Equity(445 [BWA])  Equity(926 [EGN])  \
2014-12-31 00:00:00+00:00             54.391             63.678   
2015-01-02 00:00:00+00:00             54.183             63.159   
2015-01-05 00:00:00+00:00             52.619             59.393   
2015-01-06 00:00:00+00:00             52.253             59.044   
2015-01-07 00:00:00+00:00             53.481             58.974   

                           Equity(646 [COL])  
2014-12-31 00:00:00+00:00             83.276  
2015-01-02 00:00:00+00:00             83.414  
2015-01-05 00:00:00+00:00             82.685  
2015-01-06 00:00:00+00:00             82.577  
2015-01-07 00:00:00+00:00             83.710



prices_df.shape

(253, 587)

code:

import numpy as np 
import pandas as pd

from pandas import Timestamp
import scipy.optimize as sco


mean_returns = prices_df.mean()
cov_matrix = prices_df.cov()
risk_free_rate = 0.0

def portfolio_annualised_performance(weights, mean_returns, cov_matrix):
    returns = np.sum(mean_returns*weights ) *252
    std = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(252)
    print('returns: '+str(returns))
    print('std: '+str(std))
    return std, returns

def neg_sharpe_ratio(weights, mean_returns, cov_matrix, risk_free_rate):
    p_var, p_ret = portfolio_annualised_performance(weights, mean_returns, cov_matrix)
    return -(p_ret - risk_free_rate) / p_var

def max_sharpe_ratio(mean_returns, cov_matrix, risk_free_rate):
    num_assets = len(mean_returns)
    args = (mean_returns, cov_matrix, risk_free_rate)
    constraints = ({'type': 'eq', 'fun': lambda x: np.sum(x) - 1})
    bound = (0.0,1.0)
    bounds = tuple(bound for asset in range(num_assets))
    result = sco.minimize(neg_sharpe_ratio, num_assets*[1./num_assets,], args=args,
                        method='SLSQP', bounds=bounds, constraints=constraints)
    return result






# getting asset allocations from code above
opt_results = max_sharpe_ratio(mean_returns, cov_matrix, risk_free_rate)


weights=opt_results.x

update:

I added

def max_sharpe_ratio(mean_returns, cov_matrix, risk_free_rate):
    num_assets = len(mean_returns)
    args = (mean_returns, cov_matrix, risk_free_rate)
    constraints = ({'type': 'eq', 'fun': lambda x: np.sum(x) - 1})
    bound = (0.0,1.0)
    bounds = tuple(bound for asset in range(num_assets))
    result = sco.minimize(neg_sharpe_ratio, num_assets*[1./num_assets,], args=args,
                        method='SLSQP', bounds=bounds, constraints=constraints,options={"disp": True})
    return result

and got output message:

Iteration limit exceeded    (Exit mode 9)
            Current function value: nan
            Iterations: 101
            Function evaluations: 60489
            Gradient evaluations: 101

and when I change it to:

def max_sharpe_ratio(mean_returns, cov_matrix, risk_free_rate):
    num_assets = len(mean_returns)
    args = (mean_returns, cov_matrix, risk_free_rate)
    constraints = ({'type': 'eq', 'fun': lambda x: np.sum(x) - 1})
    bound = (0.0,1.0)
    bounds = tuple(bound for asset in range(num_assets))
    result = sco.minimize(neg_sharpe_ratio, num_assets*[1./num_assets,], args=args,
                        method='SLSQP', bounds=bounds, constraints=constraints,options={"disp": True,'maxiter':1000})
    return result

I get:

Iteration limit exceeded    (Exit mode 9)
            Current function value: nan
            Iterations: 1001
            Function evaluations: 599589
            Gradient evaluations: 1001

What is the status returned by SLSQP? Also, note that this problem is often solved as a QP (after a somewhat nontrivial reformulation). — Erwin Kalvelagen, Sep 18 '20 at 07:13
@ErwinKalvelagen thank you for getting back to me. I added options={'disp':True} so I could see more detail on the output. it's returning status 9, and I've tried increasing the iterations all the way up to 1000 with no luck. do you see what the issue might be, there aren't any nan values in mean_returns or cov_matrix and it works for much smaller dataframe. — user3476463, Sep 19 '20 at 18:58
I don't trust this setup at all. `mean_returns = prices_df.mean()` actually calculates mean prices and not mean returns. There seem to be some really basic misunderstandings going on here. — Erwin Kalvelagen, Sep 20 '20 at 14:35

scipy optimization model returns all nan solutions

0 Answers0