For the following toy example, I am attempting to parallelize some nested for loops using dask delayed/compute. Is there any way I can visualize the task graph for the following?
import time
from dask import compute, delayed
@delayed
def child(val):
time.sleep(1)
return val
@delayed
def p1(val):
futs = []
for i in range(5):
futs += [child(val * i)]
return compute(*futs)
@delayed
def p2(val):
futs = []
for i in range(10):
futs += [p1(val * i)]
return compute(*futs)
@delayed
def p3(val):
futs = []
for i in range(30):
futs += [p2(val * i)]
return futs
if __name__ == "__main__":
f = p3(10)
f.visualize()
For example, when I call the .visualize
method on any of the delayed functions it returns just one level(node?) but none of the previous branches and functions. For instance p3(10).visualize()
returns
Perhaps I am using dask.delayed
improperly here?