Is there a way with dask to have a variable that can be retrieved from one task to another. I mean a variable that I could lock in the worker and then retrieve in the same worker when i execute another task.
Asked
Active
Viewed 404 times
1 Answers
1
The workers themselves are just Python processes, so you could do tricks with globals().
However, it is probably cleaner to emit values and pass these between tasks. Dask retains the right to rerun functions and run them on different machines, so depending on global state or worker-specific state can easily get you into trouble.
MRocklin
- 55,641
- 23
- 163
- 235
-
Thx for the answer but the globals tricks seems not working. I tried to run this code in my workers: `i = 0 def engine(cmd): global i print('i is :%d'%(i)) i = i + 1`. But the counter remains to 0 – Bertrand Aug 05 '16 at 12:34
-
Yeah, I recommend not using globals – MRocklin Aug 05 '16 at 19:32
-
Thx for your answers i'll try to handle that differently so. – Bertrand Aug 05 '16 at 19:56