0

I'm having a problem with profiling JVM for hadoop using Java agent as in https://github.com/etsy/statsd-jvm-profiler

The agent gets stack traces using management API at runtime. The -javaagent argument is specified in the mapred-site.xml file as follows: 

<configuration>

    <property>
        <name>mapreduce.map.java.opts</name>
        <value>-Xmx1536m -javaagent:myagent</value>
    </property>

    <property>
        <name>mapreduce.reduce.java.opts</name>
        <value>-Xmx3072m -javaagent:myagent</value>
    </property>

</configuration>

However, my agent cannot get any stack trace for the map or reduce tasks I defined when running my MapReduce application.

dazzphot
  • 307
  • 1
  • 3
  • 11

1 Answers1

-1

For profiling jvm one has multiple options

  • enable hadoop default profiling
  • run jstat, jmap while the mappers and reducers are running to know about jvm heap usage
thebluephantom
  • 16,458
  • 8
  • 40
  • 83
KrazyGautam
  • 2,839
  • 2
  • 21
  • 31