Running optimization process with GPU using PYTHON 3.5 and Backtrader

Question

I was giving a try to the optimization process of the Backtrader library. I see that the code run pretty well with multi-core CPU. It took around 22.352761494772228 second for the complete optimization process. But could be even faster if worked with GPU.
Hence, I would like to know how I can run the following with GPU:

#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime
import time

from backtrader.utils.py3 import range

import backtrader as bt
import backtrader.indicators as btind
import backtrader.feeds as btfeeds


class OptimizeStrategy(bt.Strategy):
    params = (('smaperiod', 15),
              ('macdperiod1', 12),
              ('macdperiod2', 26),
              ('macdperiod3', 9),
              )

    def __init__(self):
        # Add indicators to add load

        btind.SMA(period=self.p.smaperiod)
        btind.MACD(period_me1=self.p.macdperiod1,
                   period_me2=self.p.macdperiod2,
                   period_signal=self.p.macdperiod3)


def runstrat():
    args = parse_args()

    # Create a cerebro entity
    cerebro = bt.Cerebro(maxcpus=args.maxcpus,
                         runonce=not args.no_runonce,
                         exactbars=args.exactbars,
                         optdatas=not args.no_optdatas,
                         optreturn=not args.no_optreturn)

    # Add a strategy
    cerebro.optstrategy(
        OptimizeStrategy,
        smaperiod=range(args.ma_low, args.ma_high),
        macdperiod1=range(args.m1_low, args.m1_high),
        macdperiod2=range(args.m2_low, args.m2_high),
        macdperiod3=range(args.m3_low, args.m3_high),
    )

    # Get the dates from the args
    fromdate = datetime.datetime.strptime(args.fromdate, '%Y-%m-%d')
    todate = datetime.datetime.strptime(args.todate, '%Y-%m-%d')

    # Create the 1st data
    data = btfeeds.BacktraderCSVData(
        dataname=args.data,
        fromdate=fromdate,
        todate=todate)

    # Add the Data Feed to Cerebro
    cerebro.adddata(data)

    # clock the start of the process
    tstart = time.clock()

    # Run over everything
    stratruns = cerebro.run()

    # clock the end of the process
    tend = time.clock()

    print('==================================================')
    for stratrun in stratruns:
        print('**************************************************')
        for strat in stratrun:
            print('--------------------------------------------------')
            print(strat.p._getkwargs())
    print('==================================================')

    # print out the result
    print('Time used:', str(tend - tstart))


def parse_args():
    parser = argparse.ArgumentParser(
        description='Optimization',
        formatter_class=argparse.RawTextHelpFormatter,
    )

    parser.add_argument(
        '--data', '-d',
        default='2006-day-001.txt',
        help='data to add to the system')

    parser.add_argument(
        '--fromdate', '-f',
        default='2006-01-01',
        help='Starting date in YYYY-MM-DD format')

    parser.add_argument(
        '--todate', '-t',
        default='2006-12-31',
        help='Starting date in YYYY-MM-DD format')

    parser.add_argument(
        '--maxcpus', '-m',
        type=int, required=False, default=0,
        help=('Number of CPUs to use in the optimization'
              '\n'
              '  - 0 (default): use all available CPUs\n'
              '  - 1 -> n: use as many as specified\n'))

    parser.add_argument(
        '--no-runonce', action='store_true', required=False,
        help='Run in next mode')

    parser.add_argument(
        '--exactbars', required=False, type=int, default=0,
        help=('Use the specified exactbars still compatible with preload\n'
              '  0 No memory savings\n'
              '  -1 Moderate memory savings\n'
              '  -2 Less moderate memory savings\n'))

    parser.add_argument(
        '--no-optdatas', action='store_true', required=False,
        help='Do not optimize data preloading in optimization')

    parser.add_argument(
        '--no-optreturn', action='store_true', required=False,
        help='Do not optimize the returned values to save time')

    parser.add_argument(
        '--ma_low', type=int,
        default=10, required=False,
        help='SMA range low to optimize')

    parser.add_argument(
        '--ma_high', type=int,
        default=30, required=False,
        help='SMA range high to optimize')

    parser.add_argument(
        '--m1_low', type=int,
        default=12, required=False,
        help='MACD Fast MA range low to optimize')

    parser.add_argument(
        '--m1_high', type=int,
        default=20, required=False,
        help='MACD Fast MA range high to optimize')

    parser.add_argument(
        '--m2_low', type=int,
        default=26, required=False,
        help='MACD Slow MA range low to optimize')

    parser.add_argument(
        '--m2_high', type=int,
        default=30, required=False,
        help='MACD Slow MA range high to optimize')

    parser.add_argument(
        '--m3_low', type=int,
        default=9, required=False,
        help='MACD Signal range low to optimize')

    parser.add_argument(
        '--m3_high', type=int,
        default=15, required=False,
        help='MACD Signal range high to optimize')

    return parser.parse_args()


if __name__ == '__main__':
    runstrat()

The sample data used is here: Sample Data file

Let me know what improment I can make. I thought of using numba or Pycuda or PyOpenCL

To be able to run the code on GPU you need to make sure that the code is suited for parallel running. Otherwise it will actually be slower. So why not just try threading or multiprocessing first to see how much improvement you could get first? — hcheung, May 18 '18 at 07:54
Unless you are willing and able to completely re-write the internals of the computational library you are using, then you can't — talonmies, May 23 '18 at 09:59

score 4 · Accepted Answer · edited Jun 20 '20 at 09:12

4

I don't think this is possible for Backtrader as mentioned in their FAQs

Optimization is slow and consumes RAM

Indeed. Parameter overfitting is also a great and formidable enemy of algorithmic trading.

I want to use the GPU for optimization

Nice idea, but the multiprocessing module in Python won't do that.

https://community.backtrader.com/topic/381/faq

Cuda and all can be used in cases where you want to offload a computation to the GPU. Like below example

import numpy as np
from timeit import default_timer as timer
from numba import vectorize

@vectorize(['float32(float32, float32)'], target='cuda')
def pow(a, b):
    return a ** b

def main():
    vec_size = 100000000

    a = b = np.array(np.random.sample(vec_size), dtype=np.float32)
    c = np.zeros(vec_size, dtype=np.float32)

    start = timer()
    c = pow(a, b)
    duration = timer() - start

    print(duration)

if __name__ == '__main__':
    main()

PS: Example credits to https://weeraman.com/put-that-gpu-to-good-use-with-python-e5a437168c01

edited Jun 20 '20 at 09:12

Community

1
1

answered May 18 '18 at 08:25

Tarun Lalwani

142,312
9
204
265

This is a good one but is not what I am expecting as answer to my bounty question. lease can you help me with BackTrader module instead of presenting some different module. Can you give a try? – Jaffer Wilson May 18 '18 at 09:52
What I meant to say was that it is a big library and not built with Cuda support, so that example was just to show where cuda is used and how – Tarun Lalwani May 18 '18 at 10:35
So, that means we can't do anything for that, I guess. – Jaffer Wilson May 18 '18 at 10:36
Do you have anything else to say? As my bounty is soon expiring and I still do not have the answer to what I am looking for. Did you just posted answer because you were in need of the bounty rewards? Because your answer isn't helpful to me. Please let me know. – Jaffer Wilson May 24 '18 at 08:11
No. I know this is not a solution. But it's more like that your should use a different library or build your own with GPU support – Tarun Lalwani May 24 '18 at 08:14
Can you help me make this one GPU supported? If not then your answer is not the answer o the question but more like a comment. – Jaffer Wilson May 24 '18 at 08:15
@JafferWilson, I am afraid that is out of my zone. I am ok with you not accepting this answer and not awarding the bounty. But I would leave it as answer as it represents a fact – Tarun Lalwani May 24 '18 at 08:46
Thank you. I was expecting something else in my answer but still I will accept your fact. – Jaffer Wilson May 25 '18 at 04:09

score 0 · Answer 2 · answered May 24 '18 at 20:21

It cannot be done without rewriting the framework (as opposed to rewriting your code) One of the main goals of the platform was to be pure Python and only use packages which are in the standard library.

Reference: myself (parent of the beast)

The question has been asked before and that's why there is a explicit mention in the FAQ.

Running optimization process with GPU using PYTHON 3.5 and Backtrader

2 Answers2