1

I have a standalone spark job and every time the job finishes, the below warning occurs: I don't really understand the meaning of this and also how to solve this. Would be great if you could help. Thanks

 WARN  [SparkWorker-0 error logger] 2016-10-08 10:18:33,395 SparkWorker-0 ExternalLogger.java:92 
- Association with remote system [akka.tcp://sparkExecutor@10.47.183.30:39422] has failed, address is now gated for [5000] ms. Reason is: [Disassociated].
    INFO  [sparkMaster-akka.actor.default-dispatcher-4] 2016-10-08 10:18:33,406  Logging.scala:59 - Removing executor app-20161008101807-0002/5 because it is EXITED
    INFO  [sparkMaster-akka.actor.default-dispatcher-4] 2016-10-08 10:18:33,407  Logging.scala:59 - Launching executor app-20161008101807-0002/6 on worker worker-20161008093556-10.47.183.121-41649
    WARN  [sparkMaster-akka.actor.default-dispatcher-4] 2016-10-08 10:18:33,762  Logging.scala:71 - Got status update for unknown executor app-20161008100608-0001/4
    INFO  [sparkMaster-akka.actor.default-dispatcher-4] 2016-10-08 10:18:33,819  Logging.scala:59 - akka.tcp://sparkDriver@XXX.196.201.23:36340 got disassociated, removing it.
    INFO  [SparkWorker-0 logger] 2016-10-08 10:18:33,835 SparkWorker-0 ExternalLogger.java:88 - Executor app-20161008100608-0001/0 finished with state KILLED exitStatus 143
    WARN  [sparkMaster-akka.actor.default-dispatcher-5] 2016-10-08 10:18:33,837  Logging.scala:71 - Got status update for unknown executor app-20161008100608-0001/0
peter
  • 674
  • 1
  • 12
  • 33

1 Answers1

0

This is just the executor saying it can not talk to anyone. I would check connection ports and the like on your firewall.

mando222
  • 543
  • 3
  • 6
  • 16