0

How can map-reduce job generate metrics about how many keys it has processed and give data like the following?

% of keys that belonged to this particular value.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
adarshhsingh
  • 61
  • 1
  • 1
  • 6

1 Answers1

0

Use a custom counter in reduce method and increment it every-time the reduce method is called. Please note that the reduce method is called only once for a given key. Once the job is completed successfully, this custom counter will appear in the log.

Marco99
  • 1,639
  • 1
  • 19
  • 32
  • there is already the reduce_input_groups counter which does exactly that, but this is not I think what the OP asks, even if the question is not very clear – vefthym Feb 19 '16 at 00:04