0

Essentially I'm trying to use the brentq method row-wise on a pandas dataframe to get the root of a function that takes as arguments columns and constants as well.

Something like the following

import pandas as pd
import numpy as np

from scipy.optimize import brentq

np.random.seed(0)
df = pd.DataFrame(np.random.rand(10, 3), columns=list('ABC'))

CONST = 0.5

def fcn(x, y, k):
    return x**2 + y - k

def objective_function(x, y, k, delta):
    return fcn(x, y, k) - delta

def get_root(x, k, delta, a=-10.0, b=10.0, xtol=1e-6):

    # avoid mirroring outer scope
    _x, _k, _delta = x, k, delta

    # nested function that takes the target param as the input
    def nfcn(y):

        # get y that makes fcn and delta equal
        return objective_function(_x, y, _k, _delta)

    result = brentq(nfcn, a=a, b=b, xtol=xtol)

    return result

I got this to work with apply and lambda function, on rows

df['D'] = df.apply(lambda x: get_root(x['A'], CONST, x['C'], a=-10., b=10.), axis=1)

but when the dataframe is really big is very slow, as expected.

Any ideas on how to vectorize this?

Many thanks

AleVis
  • 187
  • 1
  • 1
  • 10

1 Answers1

1

Perhaps you can use a partial. I get a speed-up of almost 4x for a dataframe of 100k rows.

from functools import partial

# Rearrange parameters (swap `delta` and constant `k`).
def get_root(x, delta, k, a=-10.0, b=10.0, xtol=1e-6):
    ...

p = partial(get_root, k=CONST, a=-10., b=10.)
df['D'] = [p(*vals) for vals in df[['A', 'C']].values]
Alexander
  • 105,104
  • 32
  • 201
  • 196
  • Thanks. I did get the considerable speed up on single dataframes. I noticed that when iterating through a dictionary of dataframes and over the columns of the dataframes to get the .values the performance is pretty much the same though. Any thoughts on how that might be? – AleVis Nov 14 '19 at 17:06