Spark runs endlessly for Pi example

Question

I just setup Spark and ran the command

spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m

However, it just keeps endlessly printing out messages like

16/04/25 17:34:46 INFO Client: Application report for application_1460481694166_0125 (state: ACCEPTED)

I read somewhere that I could try to kill the application. But I'm not sure what

When I try

yarn application -list

I see

            Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                        Tracking-URL
application_1460481694166_0118  org.apache.spark.examples.SparkPi                     SPARK           root         default                ACCEPTED               UNDEFINED                   0%                                 N/A
application_1460481694166_0124           Spark shell                   SPARK          root         default                ACCEPTED               UNDEFINED                   0%                                 N/A
application_1460481694166_0120           Spark shell                   ...
          Zeppelin                   SPARK      zeppelin         default                 RUNNING               UNDEFINED                  10%               http://10.0.2.15:4040
application_1460481694166_0117  org.apache.spark.examples.SparkPi                     SPARK           root         default                ACCEPTED               UNDEFINED                   0%                                 N/A
application_1460481694166_0123           Spark shell

...

I'm not sure why Zeppelin is showing up because I closed it in my web browser

What do I need to do now?

score 1 · Answer 1 · answered Apr 25 '16 at 17:54

I'm guessing Zeppelin is still running even though you closed your browser. Closing the browser is not the same as stopping the hosting process. Stopping the hosting process is done in the CLI tab that started the process. As a last ditch, you can yarn application -kill any of the running processes in any tab.

yarn application -kill application_1460481694166_0118

That will kill the (first) spark application.

Spark runs endlessly for Pi example

1 Answers1