In django-q we have recycle which is The number of tasks a worker will process before recycling . Useful to release memory resources on a regular basis.
When I start dask-worker with --nprocs 2, I get two worker subprocesses.
I would like to recycle them after each of them complete 1 task because there are a lot of resources that need to be freed and removed from memory after task is complete and I have no way of freeing them in an automatic fashion.
I tried a worker plugin with a simple sys.exit() inside release_key but it doesn't get rid of the process.
How to recycle worker subprocesses after task is done?
