4

We want to use provisioned IOPS EBS volume on our mongo machines, I'm a bit confused on how to calculate the number of provisioned IOPS to use.

Currently we're getting an average of 25 IOPS (read + write) per volume (we use 4 EBS volumes combined), we're also seeing that our CPU is in 'waiting' state during the peak IO times.

I can split the question into two parts -

  1. Will we see a major increase in the average IOPS after moving to the provisioned EBS IOPS, because the IO latency will decrease, should we take a big buffer to avoid exceeding the IOPS count?
  2. How would you calculate the optimal number of provisioned IOPS for your mongo setup?

Thanks!

Edit - Attached the volume average queue length from amazon

(I can't paste the image so here's a link)

https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/13432/91327/Mz2erjjMUcxMmDN/upload.png

Nimrod Gutman
  • 143
  • 1
  • 5

1 Answers1

2

IOStat numbers are what you really need to gauge an impact here, preferably over time and relatable to MongoDB load (so I would recommend MMS with the munin-node plugin for monitoring). If you are not seeing high levels of latency on svctime and queue size (and if you are you would probably not have idle CPU, you would be seeing IOWait spike).

The general consensus from the various analyses out there has been this:

  • You will get somewhat improved latency with PIOPS (this will vary by size of workload and other factors, see here for example)
  • Generally, you will get much more consistent performance in terms of latency and throughput, although this is not always true (same report as above)
  • However, if you are not seeing any stress in terms of IO, you are unlikely to see a huge difference by using PIOPS (besides a more consistent performance)

Since your IO does not appear to be your bottleneck (I am going on limited evidence/information here) I don't think you are going to see much more than the 100 PIOPS you currently see, but you should have more predictable performance.

Adam C
  • 5,222
  • 2
  • 30
  • 52
  • 1
    Thanks for the answer - just before accepting it a couple more questions. Your assumption that our volumes aren't under load is based on the fact that the standard EBS IOPS amazon gives is ~100, right? Also I attached the amazon volume avg queue length graph, does it confirm this assumption? – Nimrod Gutman Dec 03 '13 at 08:33
  • You didn't mention the instance size but I would expect single volumes to be able to do significantly more than 25 IOPS a piece (with 4 total you get the 100 figure I mentioned) - if you were seeing IOWait, high queues (and you don't seem to be) then I would expect a significant improvement. Without obvious IOPS related stress, the biggest improvement you will probably see is a reduction in peak latency (PIOPS performance is more predictable). You may get more bang for your buck getting more RAM so MongoDB can fit more data into memory and hit disk less often (depends on your usage pattern) – Adam C Dec 03 '13 at 09:11