I recently upgraded dask-0.15.3 to dask-0.16.0 and distribute-1.19.1 to distribute-1.20.2. After upgrade all dask jobs are failing with exception: _pickle.UnpicklingError: state is not a dictionary
Please let me know if I am missing any library.
Here is the detailed exception:
File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/dask/dataframe/core.py", line 2335, in set_index
[ worker demo4-dn-1 ] : divisions=divisions, **kwargs)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/dask/dataframe/shuffle.py", line 64, in set_index
[ worker demo4-dn-1 ] : divisions, sizes, mins, maxes = base.compute(divisions, sizes, mins, maxes)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/dask/base.py", line 333, in compute
[ worker demo4-dn-1 ] : results = get(dsk, keys, **kwargs)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/client.py", line 1999, in get
[ worker demo4-dn-1 ] : results = self.gather(packed, asynchronous=asynchronous)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/client.py", line 1437, in gather
[ worker demo4-dn-1 ] : asynchronous=asynchronous)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/client.py", line 592, in sync
[ worker demo4-dn-1 ] : return sync(self.loop, func, *args, **kwargs)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/utils.py", line 254, in sync
[ worker demo4-dn-1 ] : six.reraise(*error[0])
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/six.py", line 693, in reraise
[ worker demo4-dn-1 ] : raise value
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/utils.py", line 238, in f
[ worker demo4-dn-1 ] : result[0] = yield make_coro()
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/tornado/gen.py", line 1055, in run
[ worker demo4-dn-1 ] : value = future.result()
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/tornado/concurrent.py", line 238, in result
[ worker demo4-dn-1 ] : raise_exc_info(self._exc_info)
[ worker demo4-dn-1 ] : File "", line 4, in raise_exc_info
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/tornado/gen.py", line 1063, in run
[ worker demo4-dn-1 ] : yielded = self.gen.throw(*exc_info)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/distributed/client.py", line 1315, in _gather
[ worker demo4-dn-1 ] : traceback)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/miniconda3/envs/py361/lib/python3.6/site-packages/six.py", line 692, in reraise
[ worker demo4-dn-1 ] : raise value.with_traceback(tb)
[ worker demo4-dn-1 ] : File "/ebs/d1/agent/conda/envs/py361/lib/python3.6/site-packages/distributed/protocol/pickle.py", line 59, in loads
[ worker demo4-dn-1 ] : return pickle.loads(x)
[ worker demo4-dn-1 ] : _pickle.UnpicklingError: state is not a dictionary
