0

I am trying to get workers to output some information from their ipython kernel and execute various commands in the ipython session. I tried the examples in the documentation and the ipyparallel example works, but not the second example (with ipython magics). I cannot get the workers to execute any commands. For example, I am stuck on the following issue:

from dask.distributed import Client

client = Client()

info = client.start_ipython_workers()

list_workers = info.keys()

%remote info[list_workers[0]]

The last line returns an error:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-19-9118451af441> in <module>
----> 1 get_ipython().run_line_magic('remote', "info['tcp://127.0.0.1:50497'] worker.active")

~/miniconda/envs/dask/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2334                 kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2335             with self.builtin_trap:
-> 2336                 result = fn(*args, **kwargs)
   2337             return result
   2338 

~/miniconda/envs/dask/lib/python3.7/site-packages/distributed/_ipython_utils.py in remote_magic(line, cell)
    115     info_name = split_line[0]
    116     if info_name not in ip.user_ns:
--> 117         raise NameError(info_name)
    118     connection_info = dict(ip.user_ns[info_name])
    119 

NameError: info['tcp://127.0.0.1:50497']

I would appreciate any examples of how to get any information from the ipython kernel running on workers.

SultanOrazbayev
  • 14,900
  • 3
  • 16
  • 46
  • if you first do `worker=info[list_workers[0]]`, and then `%remote worker ...`, you get one step further, but I suspect things still don't work. – mdurant Feb 19 '21 at 17:13
  • Thank you for commenting! Yes, I tried that and also tried registering custom magic with `magic_names` option in `start_ipython_workers`. I can see that the magics are created but when I try to use them I get `TimeoutError: Timeout waiting for IPython output`. – SultanOrazbayev Feb 19 '21 at 17:31
  • Please raise an issue, if there isn't one already. It may be that this functionality has been abandoned. – mdurant Feb 19 '21 at 17:42
  • Ah, ok, will do, although this is more of a curiosity for me. – SultanOrazbayev Feb 19 '21 at 17:43

1 Answers1

0

Posting here just for keeping track, I raised an issue for this on GitHub: https://github.com/dask/distributed/issues/4522

SultanOrazbayev
  • 14,900
  • 3
  • 16
  • 46