4

I'm struggling to figure out how to get dask delayed to work on a particular workflow that involves creating a dictionary.

The idea here is that func1, func2, func3 can run independently of each other at the same time, and I want the results of those functions to be the values in a new dictionary z.

from dask.delayed import delayed

x1 = {'a': 1, 'b': 2, 'c': 3}
x2 = {'a': 4, 'b': 5, 'c': 6}

@delayed
def func1(d1, d2):
    return d1['a'] + d2['a']

@delayed
def func2(d1, d2):
    return d1['b'] - d2['b']

@delayed
def func3(d1, d2):
    return d1['c'] * d2['c']

z = {}
z['val1'] = func1(x1, x2)
z['val2'] = func2(x1, x2)
z['val3'] = func3(x1, x2)

When I run the following, I get an error:

>>> result_dict = z.compute()

AttributeError: 'dict' object has no attribute 'compute'

When I run the following, it succeeds however the result is a tuple and not a dictionary.

>>> result_dict = dask.compute(z)

({'val1': 5, 'val2': -3, 'val3': 18},)

How do I compute the results so it returns a dictionary? Am I doing this right?

blahblahblah
  • 2,299
  • 8
  • 45
  • 60

1 Answers1

6

As you saw, dask.compute returns a tuple of results

>>> dask.compute(z)
({'val1': 5, 'val2': -3, 'val3': 18},)

This is because you can give it many arguments and it results a result for each argument

>>> result_dict = dask.compute(x, y, z)
(..., ..., ...)

You don't care about this, you only want the first element. You can get the first element of a tuple in Python using the getitem [...] syntax.

>>> dask.compute(z)[0]
{'val1': 5, 'val2': -3, 'val3': 18}
MRocklin
  • 55,641
  • 23
  • 163
  • 235
  • Thanks for the explanation. Just to be clear, are `func1`, `func2`, and `func3` all being executed concurrently (in other words, am I doing dask right here?)? I'm unable to run `z.visualize()` as I get a `'dict' object has no attribute 'visualize'` error to see what is happening under the hood. – blahblahblah Feb 10 '18 at 19:40
  • What you're doing is fine. The dask methods (like `.compute()`) aren't working because these aren't dask objects, they're normal Python dictionaries. You can similarly call `dask.visualize(z)` to get the image of the graph in this situation. – MRocklin Feb 10 '18 at 19:42