HighLevelGraph with (local/multiprocessing) distributed

Question

How should I use dask.highlevelgraph.HighLevelGraph in a local distributed setting.

Sequential computation e.g.

result = dask.get(some_high_level_graph, [some_targets])

works.

import dask
from dask.highlevelgraph import HighLevelGraph as CG

# from dask import distributed as ddist

def inc(i):
    return i + 1

def add(a, b):
    return a + b

def mult(a, b):
    return a * b

wf = CG(
    layers = {
        1: {
            'inc': (inc, 1),
            'add': (add, 1, 'inc'),
        },
        2: {
            'mult': (mult, 3, 3)
        }
    },
    dependencies=[1, 2]
)

targets = ['add', 'mult']
r = dask.get(wf, targets)
print(r)

targets = ['mult']
r = dask.get(wf, targets)
print(r)

Practically, I want to compute 'add' and 'mult' in parallel.

Answer 1

I was able to solve the problem myself. Just use:

result = client.get(graph, targets)