2

What is the best way to pass a Future to a Dask Delayed function such that the Future stays in tact? In other words, how can we ensure the function will get the actual Future and not the result it represents?

jakirkham
  • 685
  • 5
  • 18

1 Answers1

1

Generally the semantics are that dask.delayed functions get concrete results rather than dask-y ones. This is not easily supported today without some tricks.

That being said, I recommend the following trick:

Place your future in a Variable

import dask
from dask.distributed import Client, Variable
client = Client()

v = Variable()

@dask.delayed
def f(v):
    return v.get().result()

future = client.scatter(123)
f(future).compute()
# 123
MRocklin
  • 55,641
  • 23
  • 163
  • 235
  • Thanks. That does seem reasonable. Also found that `dask.core.literal` worked, would that be a reasonable thing to use? – jakirkham Mar 12 '18 at 15:33
  • Hrm, yes, that seems like a better idea. I recommend raising a PR to push that into the top-level API and pinging jcrist to see what his thoughts are. – MRocklin Mar 12 '18 at 15:37