6

I have the Dask code below that submits N workers, where each worker is implemented in a Docker container:

default_sums = client.map(process_asset_defaults, build_worker_args(req, numWorkers))
future_total_sum = client.submit(sum, default_sums)
total_defaults_sum = future_total_sum.result()

where process_asset_defaults is a method in a worker.

The problem is that in a development environment when I change the worker's code I need to restart all the containers manually for the change to take effect.

Is there a way to reload the worker with the new code without restarting the workers?

Note: the code resides in a Docker volume, I change it directly in the volume with Visual Studio Code.

ps0604
  • 1,227
  • 23
  • 133
  • 330

2 Answers2

1

You could make make a variable with the current content of the file, then make a loop which checks if the file content is not equals to the one before, and if it is not you can do stuff

before = open("/code/app/worker.py").read()

while True:
   current = open("/code/app/worker.py").read()
   if current != before:
       dostuff()
       before = current
Robiot
  • 98
  • 10
  • How do I link your code with loading the worker `client.upload_file('/code/app/worker.py') ` ? Even when I run `upload_file` more than once I need to restart the container. – ps0604 Jul 02 '21 at 20:45
1

If the code that changes is the content of the functions, then it might be possible to use autoreload:

import importlib
importlib.reload(process_asset_defaults) # if this is the function that needs updating

See blog or docs for some further details.

SultanOrazbayev
  • 14,900
  • 3
  • 16
  • 46
  • Thanks, where should `reload` be invoked? right after `client.upload_file`? What if the function calls other functions? – ps0604 Jul 04 '21 at 15:03
  • It's tough to say without seeing the full pipeline, if your workers receive function definitions from client, then I guess invoking it before `client.submit` should be sufficient... but if your workers use some other code also, then their definitions should also be refreshed... – SultanOrazbayev Jul 07 '21 at 04:21
  • Thanks for your answer. I'm getting `TypeError: reload() argument must be a module` any ideas? I added the import with no issues. – ps0604 Jul 09 '21 at 11:59
  • This is how I import the function: `from ...worker.worker_asset_defaults import process_asset_defaults` – ps0604 Jul 09 '21 at 12:39
  • Not 100% sure, but in the example you gave you would probably want to reload the module `worker.worker_asset_defaults` (I don't think autoreload is able to track where a function came from, so it reloads only the whole modules, not individual functions). – SultanOrazbayev Jul 09 '21 at 17:56