2

I am a new user to try dask delayed. I want to use delayed to automatically transform function and code into Delayed. However, I found delayed.compute didn't recursively compute Delayed in collection... 

from dask import delayed, base

@delayed
def inc(x):
    return x + 1

@delayed
def colls(ind):
    return [inc(i) for i in xrange(ind)]


data2 = colls(2)
data2.compute() # I expect [1, 2], but get [Delayed('inc-...'),
 Delayed('inc-...')]

Did I missing any thing to make it work or Dask.delayed doesn't support it?

Colin Yu
  • 23
  • 2

1 Answers1

1

You are correct that you should not use delayed functions within other delayed functions (unless you are doing something very strange). However you can pass delayed values into other delayed functions.

In your particular example I would leave colls as not-delayed. You want it to immediately determine how many delayed inc calls to make. Generally you want to immediately call any code that builds out your task graph and delay any function that just does work.

from dask import delayed, compute

@delayed
def inc(x):
    return x + 1

def colls(ind):
    return [inc(i) for i in xrange(ind)]


data2 = colls(2)
compute(data2)
# [1, 2]
MRocklin
  • 55,641
  • 23
  • 163
  • 235
  • Thank you. Actually, I want to use dask.delayed to do something cool and strange... I want to add delayed to wrapper codeblock in the codebase to make Delayeds in everywhere... However, it is another topic. – Colin Yu Jun 10 '17 at 08:44