0

I'm trying to start a dask cluster over SSH, but I am encountering a strange errors like these:

Exception in thread Thread-6:
Traceback (most recent call last):
  File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/localuser/miniconda3/lib/python3.6/site-packages/distributed/deploy/ssh.py", line 57, in async_ssh
    banner_timeout=20)  # Helps prevent timeouts when many concurrent ssh connections are opened.
  File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 329, in connect
    to_try = list(self._families_and_addresses(hostname, port))
  File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 200, in _families_and_addresses
    hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
  File "/home/localuser/miniconda3/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

I'm starting the cluster like this:

$ dask-ssh --ssh-private-key ~/.ssh/cluster_id_rsa \
      --hostfile ~/dask-hosts.txt \
      --remote-python "~/miniconda3/bin/python3.6"

My dask-hosts.txt looks like this:

localuser@127.0.0.1
remoteuser@10.10.4.200
...
remoteuser@10.10.4.207

I get the same error with/without the localhost line.

I have checked the ssh setup, I can login to all the nodes using a public key setup (the key is unencrypted, to avoid decryption prompts). What am I missing?

suvayu
  • 4,271
  • 2
  • 29
  • 35

1 Answers1

1

The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:

–hostfile PATH Textfile with hostnames/IP addresses

You can use --ssh-username to set a username (although only a single one).

fuglede
  • 17,388
  • 2
  • 54
  • 99
  • That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names. – suvayu Nov 12 '18 at 07:00
  • Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on https://github.com/dask/distributed/issues. – fuglede Nov 12 '18 at 07:30