1

I am running a Dask script on a EC2 instance from AWS. I would like to connect and see the dashboard provided by Dask, but I can't figure out how.

I am creating a Local Cluster on my EC2 instance, the scripts runs fine and I am connecting to my AWS instance via Putty. However, I would like to see the available dashboard: on my PC it is enough to connect to the provided IP and port, but I am not able to do that on the AWS machine.

Once the script is running, this is my output for the "parameters" of the local cluster:

<Client: 'inproc://172.31.29.4/7475/1' processes=1 threads=8, memory=27.94 GiB>
LocalCluster(b8be08dd, 'inproc://172.31.29.4/7475/1', workers=1, threads=8, memory=27.94 GiB)
dashboard address: {'dashboard': 8787}

For example, I tried to write 172.32.29.4:8787/status on my browser, but I wasn't able to connect to the dashboard.

I already checked this question: How to view Dask dashboard when running on a virtual machine? However I am using a Local Cluster and I would like to connect to its dashboard from remote. Is it possible? If so, how?

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Mattia Surricchio
  • 1,362
  • 2
  • 21
  • 49
  • 1
    172.31.29.4 is a local IP address, like 192.168.1.1. You can't use that address from outside the machine's subnet. You would need a public IP address. – Tim Roberts Apr 18 '21 at 06:30
  • 1
    Get the [Public IP](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html#concepts-public-addresses) of that EC2 and then ensure the EC2 [Security Group allows port 8787](https://ec2-tutorials.readthedocs.io/en/latest/configure-firewall.html). While all this is asked, DO NOTE: Its not safe to allow access to all via Security Group. it would be good to add an [Auth Layer to that dashboard](https://gateway.dask.org/configuration-user.html#default-configuration). – Nagaraj Tantri Apr 18 '21 at 10:04

2 Answers2

2

The answer is in the comments, but I will type it out here, so that the original question looks "answered".

You need two things to connect to a port on an EC2 machine: the external IP, and access. The former is most easily found from the AWS console. For the latter, you typically need to edit the security group to add an inbound TCP rule for the port (either open to the world, or just your IP). There are other ways to do this part, depending on whether your machine is inside a VPC, has any custom gateways or routers... but if you don't know what that means, find the security group first. Both the public IP and the security group will be linked from the machine's row in the EC2 "running instances" list.

mdurant
  • 27,272
  • 5
  • 45
  • 74
  • I managed to make it work. I basically added an in-bound TCP rule in my AWS instance allowing the access to the 8787 port. Then i connected with Putty using the public IP (available in the instances settings) and selecting 8787 as port. Once putty is running and connected, the terminal seems dead, but it is not a problem. Open your browser and connect (with putty still running) to "public_ip_AWS:8787/status", and you will be able to see your dashboard. This might be useful for future users – Mattia Surricchio Apr 20 '21 at 11:43
0

I've setup dask-labextension visualizations to provide this type of UI.

enter image description here

Create the client object:

from dask.distributed import Client
client = Client()

Then click the magnifying glass provided by the extension to automatically connect with the cluster.

The detailed instructions are in this post.

Powers
  • 18,150
  • 10
  • 103
  • 108