1

I am running some basic examples with Giraph and I want to verify the data being read by my EdgeInputFormat. On a classic MapReduce job I could do that using Counters and Giraph uses aggregators for this. (https://giraph.apache.org/aggregators.html)

It is said on the website that the aggregators should be registered on the master. The only code I wrote is a class that extends BasicComputation and implements the compute() method. Can somebody point Where should I register them and how do I use them while reading the data?

Sorin
  • 908
  • 2
  • 8
  • 19

1 Answers1

1

I've figured out how to do it.

I have implemented a class which extends MasterCompute and in the initialize() method I registered a persistent aggregator. I used IntSumAggregator.

After that, in EdgeInputFormat I could use aggregate("myAggregator", new IntWritable(1));.

The MasterCompute was set using -mc line argument.

Sorin
  • 908
  • 2
  • 8
  • 19
  • Question: Where did you find examples of EdgeInputFormat being used? I looked quite a bit but found nothing concrete! Thank you. – Matthew Cornell Aug 11 '14 at 21:01
  • I don't remember exactly, but there might be something in the examples package. The documentation was also very useful. – Sorin Aug 18 '14 at 09:01