Having trouble starting cluster on remote machines. When starting cluster “manually” and copy json config files everything works. I can connect with notebook and everything works as expected.
But I would like to be able to start cluster over SSH.
- Host2 (2 engines)
- Host1 (Controller + 2 engines)
- Laptop - notebook
Laptop
ipython profile create --parallel --profile=ssh
ipcluster_config.py
c.IPClusterEngines.engine_launcher_class = 'SSHEngineSetLauncher'
c.IPClusterStart.controller_launcher_class = 'SSHControllerLauncher'
c.SSHControllerLauncher.hostname = 'host1'
c.SSHControllerLauncher.user = 'root'
c.SSHControllerLauncher.controller_args = ['--reuse', '--ip=*', '--profile-dir=/root/.ipython/profile_ssh']
c.SSHEngineSetLauncher.engines = {"host1":2, "host2":2}
Starting notebook
jupyter notebook --debug
When I start notebook cluster from ipython clusters tab. Engines connect to controller, files get copied etc. But when I try to connect with notebook
import ipyparallel as ipp
c = ipp.Client(profile='ssh',debug=True)
c.ids
c[:].apply_sync(lambda : "Hello, World")
“TimeoutError: Hub connection request timed out”
From console log I can see that engines are running and connected successfully.
