2

I'm going through the Apache Giraph quick start tutorial: http://giraph.apache.org/quick_start.html and have successfully setup a pseudo-distributed hadoop cluster and have successfully run the example mapreduce jobs.

However when moving to the Giraph section, I've installed Giraph using maven and I'm for some reason unable to run the simplest of Giraph jobs.

I've added the tiny_graph to my hdfs, but running their example command:

hduser@fonz-VirtualBox:~/Giraph$ $HADOOP_HOME/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 1

Gives me:

14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
14/12/18 11:19:28 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 4)
14/12/18 11:19:31 INFO job.GiraphJob: Tracking URL: http://hdnode01:50030/jobdetails.jsp?jobid=job_201412171202_0010
14/12/18 11:19:31 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers
14/12/18 11:20:59 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer fonz-virtualbox:22181 --zkNode /_hadoopBsp/job_201412171202_0010/_haltComputation'
14/12/18 11:20:59 INFO mapred.JobClient: Running job: job_201412171202_0010
14/12/18 11:21:00 INFO mapred.JobClient:  map 100% reduce 0%
14/12/18 11:30:16 INFO mapred.JobClient:  map 50% reduce 0%
14/12/18 11:30:24 INFO mapred.JobClient: Job complete: job_201412171202_0010
14/12/18 11:30:24 INFO mapred.JobClient: Counters: 6
14/12/18 11:30:24 INFO mapred.JobClient:   Job Counters 
14/12/18 11:30:24 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=1244789
14/12/18 11:30:24 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient:     Launched map tasks=2
14/12/18 11:30:24 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/12/18 11:30:24 INFO mapred.JobClient:     Failed map tasks=1

Any ideas as to why this has failed would be greatly appreciated!

Thanks very much.

Will

marsty5
  • 55
  • 9
WillJones
  • 907
  • 1
  • 9
  • 19

2 Answers2

2

The shortest path example does not have a main method and it uses hard coded source ID as 1, to start the algorithm. Therefore there is no vote to halt after 100% Map. Go to https://www.youtube.com/watch?v=yEHeJhMXJy0, and update the sssp.java file and export it to a new Jar. Then run it. I was facing the same issue and it fixed it.

Rishabh Wadhawan
  • 31
  • 3
  • Hi Rishabh, thanks for the response and the link. Followed through with what the video outlined, i.e. adding the code in the next comment to the SimpleShortestPathsVertex.java file and then rebuilding everything, but I still get the same error! – WillJones Dec 24 '14 at 17:45
  • public class SimpleShortestPathsComputation extends BasicComputation< LongWritable, DoubleWritable, FloatWritable, DoubleWritable> { **private static long startVertexId; public static void main(String[] args) { startVertexId = Long.valueOf(args[0]); }** /** The shortest paths id */ public static final LongConfOption SOURCE_ID = new LongConfOption("SimpleShortestPathsVertex.sourceId", **startVertexId**, "The shortest paths id"); /** Class logger */ private static final Logger LOG = Logger.getLogger(SimpleShortestPathsComputation.class); – WillJones Dec 24 '14 at 17:48
  • I think you have to use tool runner for running the Giraph code. But this should also work. Is your system still hanging even after implementing the main method. – Rishabh Wadhawan Jan 30 '15 at 09:52
  • @Rishabh Wadhawan could you please give me some helps to solve this problem: http://stackoverflow.com/questions/29106353/giraph-example-shortestpath-fails – Trung Bún Mar 17 '15 at 20:03
  • Can you please show me the whole screenshot of whatever you got on the terminal because what i see is that there are no errors but info's which are basically like warnings. Please tell me what are you exactly getting on the terminal. Show the whole thing. – Rishabh Wadhawan Mar 18 '15 at 00:17
  • Can you also provide me the command that you typed in to run your giraph example job. – Rishabh Wadhawan Mar 18 '15 at 00:21
0

When downloading Hadoop, download hadoop-0.20.205.0.tar.gz instead of hadoop-0.20.203.0rc1.tar.gz to avoid the Failed map tasks=1 error.

Man
  • 68
  • 5