ec2, pyspark: java.net.UnknownHostException Name or service not known

Question

I have set up the pyspark environment on AWS ec2 instance following this tutorial

However, when I simply test

sc = SparkContext.getOrCreate()

it keeps giving the error as follow:

I use 54.255.207.212 to connect my ec2, and after log in, the ip-address for this ec2 is 172.30.0.112.

Is there anything wrong about the configuration or the hostname? Thanks a lot in advance!

Share your security group configurations. I think that's where issue resides in — Rumesh Eranga Hapuarachchi, Jul 18 '17 at 04:40

score 0 · Accepted Answer · answered Jul 18 '17 at 04:44

0

Under your security group configuration, allow all inbound traffic to the instance.

answered Jul 18 '17 at 04:44

2

Its rather safe and secure to have specific port open to all, or "All Traffic" to all to specific security groups. – Vineeth Jul 18 '17 at 05:49
@Rumesh Eranga it works!!! I have surveyed some document about VPC and finally know what's wrong. Thanks a lot for help! – Christie Chen Jul 19 '17 at 08:17
@Vineeth thanks for remainder! I have configured the inbound connection to certain users concerning the safety issue. – Christie Chen Jul 19 '17 at 08:21
@陳嘉茹 if it helps, an upvote would send us some love :) – Rumesh Eranga Hapuarachchi Jul 19 '17 at 09:01

1 Answers1