I am new in Giraph and Hadoop. I am trying to run the shortest path algorithm in a multi nodes cluster (1 master and two slaves). I used the following command to run the algorithm:
bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.3.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation
-vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/hduser/input/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /user/hduser/output/shortestpaths
-w 3
-ca giraph.SplitMasterWorker=false
If I use only one worker, the job succeeds but when I increase the number of workers the job never ends:
16/12/05 20:43:42 INFO mapreduce.Job: Running job: job_local592648355_0001
16/12/05 20:43:42 INFO graph.GraphTaskManager: installGCMonitoring: name = MarkSweepCompact, action = end of major GC, cause = Permanent Generation Full, duration = 47ms
16/12/05 20:43:43 INFO mapreduce.Job: Job job_local592648355_0001 running in uber mode : false
16/12/05 20:43:43 INFO mapreduce.Job: map 33% reduce 0%
16/12/05 20:43:50 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:43:59 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:06 INFO master.BspServiceMaster: checkWorkers: Only found 1 responses of 3 needed to start superstep -1. Reporting every 30000 msecs, 569953 more msecs left before giving up.
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x27 zxid:0x17 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x28 zxid:0x18 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
16/12/05 20:44:11 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:20 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
Any ideas?? Many thanks
