Using a Spark 1.6.1 standalone cluster. After a system restart (and only minor config changes to /etc/hosts per worker) Spark executors suddenly started throwing errors that they couldn't connect to spark_master
.
When I echo $SPARK_MASTER_IP
on the same shell used to start the master, it correctly identifies the host as master.cluster
. And when I open the GUI at port 8080
it also identifies the master as Spark Master at spark://master.cluster:7077
.
I've also set in spark-env.sh
the SPARK_MASTER_IP
as well. Why are my executors trying to connect to spark_master
?