1

Used below to test the limit of what throughtput the disk can achieve

dd if=/dev/zero of=test bs=4k count=25000 conv=fdatasync

with multiple runs it averaged out to about 130 MB/s

now when running cassandra on these system i am monitoring the disk usage using

iostat -dmxt 30 sdd sdb sdc

there are certain entries i want to make sure i am interpreting them correctly like below. 

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
sdc               0.00  2718.60  186.30   27.20    17.87    12.06   287.13    44.98  215.06   2.79  59.58

even though the sum of rMB/s + wMB/s should be roughly equal to %util(disk throughput which is 130MB/s) and i am assuming some of the utilization goes towards seek , can the difference be huge enough to take about 24% of utilization.

Thanks in advance for any help.

anishek
  • 1,675
  • 2
  • 13
  • 19
  • While throughput may be 130 MB/sec, how many IO ops/sec can that disk device do? A SATA drive can normally do about 60-70 random IO operations per second. Try writing random 512-byte blocks directly to a raw disk device where the actual device is a single SATA disk. You'll be lucky to get 40 kB/sec throughput, and you'll see 100% utilization while you're doing those writes. – Andrew Henle May 11 '15 at 17:00
  • Thanks Andrew so looks like you are saying with random writes it not uncommon to get very low throughput with full utilization. But then how do i estimate the capability of my one node from a disk perspective. cassandra does only random reads, all writes are sequential. i am not able to gauge how many ops will it support with the SATA disk i have or even if i go for costlier SSD ? – anishek May 12 '15 at 05:12

1 Answers1

0

the frequent spin/seek does take significant amount of (latency) time. in my test, the io bandwidth between sequential io and random io is about 3x. also, it is better to use fio (https://github.com/axboe/fio) to run this type of tests, e.g direct io, sequential read/write with proper sector size (256kb or 512kb - depending on the support from the controller) and libaio as io engine, io queue depth 64. the test will be muchly controlled.

user3044236
  • 241
  • 3
  • 7