1

I've just started trying out AWS. I have a Postgres micro instance on RDS and I'm running a crawler on a c4.large. When I have just one spider (one thread), I get about 10 write IOPS. but if two spiders (2 threads) were deployed, I get about just 7 write IOPS. If I understand correctly, since I've allocated 20gb, the maximum IOPS should be 60IOPS in total. I've attached the monitoring below.

When only one spider was deployed: enter image description here

When a second spider was deployed alongside: enter image description here

Note the drop from an average 10 write IOPS to an average 7 write IOPS.

Any help will be much appreciated.

Kar
  • 123
  • 2
  • How do you know it is io-bound? – Peter Mar 14 '15 at 08:51
  • I get a similar situation with a larger instance, like a c4.xlarge. There, with one spider, I get about 12 write IOPS. With 2 spiders, about 24 write IOPS. With 3 spiders, it's still at 24 and with 4 spiders, it goes down to around 20. – Kar Mar 14 '15 at 10:43

1 Answers1

1

What kind of I/O backend are you using? It is a mechanical or flash (SSD) one?

In the case of mechanical backend, what you are observing is the expected behavior: Postgres is a synchronized-write heavy application, and this mean that even if you increase the running threads, the in-flight writes (queue depth) will stay low. At the same time, you are constantly forcing the disk's heads to move to (typically) very different locations.

In other word, you are issuing an heavier load on yours disk without the benefit that arise with longer queue depth.

This is the very reason because databases are often deployed in servers with a battery-backupped write back cache: the write back cache absorb multiple writes and coalesce then, enabling higher queue depth for disk operations.

shodanshok
  • 47,711
  • 7
  • 111
  • 180
  • It's a SSD (General Purpose) though. – Kar Mar 14 '15 at 10:25
  • If I remember correctly, general purpose SSD (as defined by Amazon) is really slow in terms of sustained iops. Maybe this synchronized-write scenario, with correlated low queue depth, is a bad match for that type of IO backend – shodanshok Mar 14 '15 at 12:03
  • But how is this kind of 'sustained IOPS', which is made by 2 spiders, any different to a web server being swarmed by requests writing to the db? – Kar Mar 14 '15 at 21:31
  • @Kate It's bursty. The speed is artificially regulated by Amazon; you get a "pool" of I/O operations that fills up slowly. You can use it in rapid bursts for very fast I/O, but once the pool is empty you can only use a trickle. So it's great for bursty workloads and terrible for sustained ones. You might want to consider getting better storage, like a provisioned IOPS backend. Or not using AWS, since it's frankly just really slow. – Craig Ringer Mar 16 '15 at 03:00