DASK Delayed and modules with multiple return variables

Question

I want to use delayed function from DASK. Unfortunately, Using delayed function on modules with multiple return values is not clear to me. For example, if I run following snippet, there is no way to point out to the first return value of inc module after computing final result.

from dask import delayed
from time import sleep

def inc(x):
    sleep(1)
    return x+1, x+2

def add(x, y):
    sleep(1)
    return x + y

x = delayed(inc)(1)
y = delayed(inc)(2)
z = delayed(add)(x[0], y[0])
print(z.compute())

Correctly, the result of this snippet is as follow. This indicates pointing is working.

(3, 5)

However, I can not read x return's value.

%%time
result=z.compute()
print('Result is :', result)
print(x[0])

output: 
Result is : 5
Delayed('getitem-f80cf501fe43d6cbd50e0bec71c91c12')
CPU times: user 7.46 ms, sys: 2.13 ms, total: 9.59 ms
Wall time: 2 s

It would be nice to read delayed function's return value without computing them if they are already computed because of dependency.

score 0 · Answer 1 · answered Nov 03 '20 at 19:37

By default Dask's compute method returns concrete results, but cleans up all intermediate results. This is especially important for workloads that require a lot of memory. It is important to clean up intermediate results on the fly.

If you want to compute many results at the same time, consider the dask.compute function.

x, z = dask.compute(x, z)

DASK Delayed and modules with multiple return variables

1 Answers1