1

This is what i get from the counter when i run my mapreduce jobs.

How to get other counter like CPU Time Spent, physical memory (bytes), Virtual memory (bytes), etc to be shown in the terminal?

File System Counters

    FILE: Number of bytes read=615414

    FILE: Number of bytes written=1469792

    FILE: Number of read operations=0

    FILE: Number of large read operations=0

    FILE: Number of write operations=0

    HDFS: Number of bytes read=7948610

    HDFS: Number of bytes written=216734

    HDFS: Number of read operations=13

    HDFS: Number of large read operations=0

    HDFS: Number of write operations=4

Map-Reduce Framework

    Map input records=32562

    Map output records=488415

    Map output bytes=5927964

    Map output materialized bytes=304438

    Input split bytes=110

    Combine input records=488415

    Combine output records=22055

    Reduce input groups=22055

    Reduce shuffle bytes=304438

    Reduce input records=22055

    Reduce output records=22055

    Spilled Records=44110

    Shuffled Maps =1

    Failed Shuffles=0

    Merged Map outputs=1

    GC time elapsed (ms)=346

    Total committed heap usage (bytes)=554172416

Shuffle Errors

    BAD_ID=0

    CONNECTION=0

    IO_ERROR=0

    WRONG_LENGTH=0

    WRONG_MAP=0

    WRONG_REDUCE=0

File Input Format Counters

    Bytes Read=3974305

File Output Format Counters

    Bytes Written=216734

thank you for your attention.

Andre
  • 25
  • 5
  • Your answers are in the [TaskCounter.class](http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-core/2.5.0/org/apache/hadoop/mapreduce/TaskCounter.java). And, I think you can access it using `context.getCounter` . – philantrovert Jul 10 '17 at 14:03
  • thanks a lot @philantrovert ! now i can see the other Counter! – Andre Jul 11 '17 at 01:52

0 Answers0