I am running a hadoop-based application (hadoop 1.2.1), where I had discovered a hard-disk bottleneck; monitoring the hard disk IOPS with iotop resulted in ~160 MB/sec, which was the writing speed I was getting by benchmarking the disk with dd.
Looking to eliminate the bottleneck, I mounted my whole hadoop HDFS and the mappers' output to tmpfs using:
mount -t tmpfs -o size=5G none /app/hadoop
/app/hadoop is where the HDFS gets stored, as well as the mappers' intermediate output. The input dataset on HDFS is 680 MiB, the output is 318 MiB, while the mappers' total output is 2.30 GiB; all these split across 2 datanodes/tasktrackers, thus I have adequate RAM space for the application to run and execute properly.
In this setup, I am trying to measure the IOPS to this directory, namely the writing speed. As I have read in this thread, traditional monitoring tools will not work with tmpfs, not being a block device. How could I be able to measure the writing speed?
Solutions such as using pv, as proposed in other thread, would unfortunately not work; altering the hadoop source code is not a viable option and pv introduces its own bottlenecks when writing to the RAM, as I measured with dd.
