0

I am trying to install a spark stand-alone-cluster on CentOS8 and keep getting a connection reset when trying to access the webUI of the master after I start it. Am I missing something? Help please.

I have:

  1. installed java:

openjdk version "1.8.0_302" OpenJDK Runtime Environment (build 1.8.0_302-b08) OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)

  1. Installed scala:

    Scala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL

/usr/lib/scala -> /usr/lib/scala3-3.0.1/

  1. Installed spark:

scala -> /usr/lib/scala3-3.0.1/

  1. configured bash profile with environmental variables in .bash_profile:

export PATH=$PATH/user/lib/scala/bin export SPARK_HOME=/opt/spark export PATH=$PATH:$SPARK_HOME/bin

  1. opened up some ports in the firewall:

ports: 6066/tcp 7077/tcp 8080-8081/tcp

  1. added config to spark-env.sh

SPARK_LOCAL_IP='localhost' SPARK_MASTER_IP='spark-master' SPARK_MASTER_PORT='7077'

  1. started the master using start-master.sh

  2. Using firefox to go to http://localhost:7077 - but keep getting a connection reset

dippas
  • 58,591
  • 15
  • 114
  • 126
PepperHill
  • 1

1 Answers1

0

Spark master web UI port will be 8080 by default. Check at http://localhost:8080. You can change that using SPARK_MASTER_WEBUI_PORT env variable. For your reference.

Mohana B C
  • 5,021
  • 1
  • 9
  • 28
  • Still getting a connection was reset message - 7077 is definitely open – PepperHill Aug 02 '21 at 03:17
  • Check the log file at $SPARK_HOME/logs. You should have below info lines if you master is up and running. 21/08/02 10:16:20 INFO Utils: Successfully started service 'sparkMaster' on port 7077. 21/08/02 10:16:20 INFO Master: Starting Spark master at spark://:7077 21/08/02 10:16:20 INFO Master: Running Spark version 2.4.5 21/08/02 10:16:20 INFO Utils: Successfully started service 'MasterUI' on port 8080. 21/08/02 10:16:20 INFO MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at http://:8080 21/08/02 10:16:20 INFO Master: I have been elected leader! New state: ALIVE – Mohana B C Aug 02 '21 at 04:48