Returning intermediate results from function in Python

Question

Imagine I've got a Python module with some function in it:

def sumvars(x, y, z):
    s = x
    s += y
    s += z
    return s

But sometimes I want to get results of some intermediate calculations (for example, I could have a function which reverses a matrix and would like to know the determinant which has been calculated as an intermediate step as well). Obviously, I wouldn't want to redo those calculations again if they were already done within that function.

My first idea is to return a dict:

def sumvars(x, y, z):
    d = {}
    s = x
    d['first_step'] = s
    s += y
    d['second_step'] = s
    s += z
    d['final'] = s
    return d

But I don't recall any functions in numpy or scipy which return dicts and so it seems like this might be not a good idea. (Why?) Also routinely I'll always have to type sumvars(x,y,z)['final'] for a default return value...

Another option I see is creating global variables but seems wrong having a bunch of them in my module, I would need to remember their names and in addition not being attached to the function itself looks like a bad design choice.

What would be the proper function design for such situation?

Answer 1

Put the common calculation into its own function as Jayanth Koushik recommended if that calculation can be named appropriately. If you want to return many values (an intermediate result and a final result) from a single function then a dict may be an overkill depending on what is your goal but in python it is much more natural to simply return a tuple if your function has many values to return:

def myfunc():
    intermediate = 5
    result = 6
    return intermediate, result

# using the function:
intermediate, result = myfunc()

Answer 2

Not sure if function attributes is a good idea:

In [569]: def sumvars(x, y, z):
     ...:     s = x
     ...:     sumvars.first_step = s
     ...:     s += y
     ...:     sumvars.second_step = s
     ...:     s += z
     ...:     return s


In [570]: res=sumvars(1,2,3)
     ...: print res, sumvars.first_step, sumvars.second_step
     ...: 
6 1 3

Note: as @BrenBarn mentioned, this idea is just like global variables, your previously calculated "intermediate results" could not be stored when you want to reuse them.

Answer 3

Generally when you have two different ways you want to return data, go ahead and make two different functions. "Flat is better than nested", after all. Just have one call the other so that you Don't Repeat Yourself.

For example, in the standard library, urllib.parse has parse_qs (which returns a dict) and parse_qsl (which returns a list). parse_qs just then calls the other:

def parse_qs(...):

    parsed_result = {}
    pairs = parse_qsl(qs, keep_blank_values, strict_parsing,
                      encoding=encoding, errors=errors)
    for name, value in pairs:
        if name in parsed_result:
            parsed_result[name].append(value)
        else:
            parsed_result[name] = [value]
    return parsed_result

Pretty straightforward. So in your example it seems fine to have

def sumvars(x, y, z):
    return sumvars_with_intermediates(x, y, z).final

def sumvars_with_intermediates(x, y, z):
    ...
    return my_namedtuple(final, first_step, second_step)

(I favor returning namedtuples instead of dicts from my APIs, it's just prettier)

Another obvious example is in re: re.findall is its own function, not some configuration flag to search.

Now, the standard library is a sprawling thing made by many authors, so you'll find counterexamples to every example. You'll far more often see the above pattern rather than one omnibus function that accepts some configuration flags, though, and I find it far more readable.

Answer 4

Just came up with this idea which could be a better solution:

def sumvars(x, y, z, mode = 'default'):
    d = {}
    s = x
    d['first_step'] = s
    s += y
    d['second_step'] = s
    s += z
    d['final'] = s
    if mode == 'default':
        return s
    else:
        return d

Answer 5

I belive the proper solution is to use a class, to have a better grasp of what you are modeling. For example in the case of the Matrix, you could simply store the determinant in the "determinant" attribute.

Here is an example using your matrix example.

class Matrix:
    determinant = 0

    def calculate_determinant(self):
        #calculations
        return determinant

    def some_method(self, args):
       # some calculations here

       self.determinant = self.calculate_determinant()

       # other calculations

 matrix = Matrix()
 matrix.some_method(x, y, z)
 print matrix.determinant

This also allows you to separate your method into simpler methods, like one for calculating the determinant of your matrix.

Answer 6

Another variation:

def sumvars(x, y, z, d=None):
  s = x
  if not d is None:
    d['first_step'] = s
  s += y
  if not d is None:
    d['second_step'] = s
  s += z
  return s

The function always returns the desired value without packing it into a tuple or dictionary. The intermediate results are still available, but only if requested. The call

sumvars(1, 2, 3)

just returns 6 without storing intermediate values. But the call

d = {}
sumvars(1, 2, 3, d)

returns the same answer 6 and inserts the intermediate calculations into the supplied dictionary.

Answer 7

Option 1. Make two separate functions.

Option 2. Use a generator:

>>> def my_func():
...     yield 1
...     yield 2
... 
>>> result_gen = my_func()
>>> result_gen
<generator object my_func at 0x7f62a8449370>
>>> next(result_gen)
1
>>> next(result_gen)
2
>>> next(result_gen)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

Answer 8

Inspired by @zhangxaochen solution, here's my take on your problem using class attributes:

class MyClass():
    def __init__(self):
        self.i = 4

    def f(self):
        s = self.i
        MyClass.first_step = s
        print(MyClass.first_step)

        s += self.i
        MyClass.second_step = s
        print(MyClass.second_step)

        s += self.i
        return s

def main():
    x = MyClass()
    print(x.f()) # print final s
    print(x.first_step)
    print(x.second_step)
    print(MyClass.second_step)

Note: I included several prints to make it more explicit how attribute values can be retrieved.

Result:

4
8
12
4
8
8