0

I've been testing a SAN with fibre channel disks and I get 500MBPS Writes 200MBPS sequential reads and about 5MBPS randon reads / 1000 IOps max. Any idea why reads are so much slower? It is a 3PAR SAN attached using fibre channel switches.

Seems we only had 24 Fiber drives, rated for 200 iops and they were overloading them to 300. We now have 48 drives which seems to have helped writes (2 more controller nodes with more cache) but random reads are still the same. seems it may be related to HBA / OS tuning at this point. Any ideas of what best case random iops would be from 48 FC drives?

coderwhiz
  • 212
  • 1
  • 11
  • More info. RAID structure, disk speed, testing method? – Shane Madden May 04 '11 at 03:55
  • How many disks? What type/speed of disks? What FC gear? 1 gig, 8 gig? More information is needed, please. – EEAA May 04 '11 at 03:56
  • @Shane - heh, the two of us were the only two to comment on the OP's [last question](http://serverfault.com/questions/257520/qlogic-hba-tuning-recommendations-random-iops) as well, to which there was never any response. – EEAA May 04 '11 at 04:00
  • @ErikA Haha, crazy! – Shane Madden May 04 '11 at 04:12
  • sorry guys its been crazy lately. So the sad part is its a shared SAN that we do not get 100% visibility into all of the hardware. the disks are Fiber Channel, 24 of them currently (doubling next week). HBAs are 2x Dual Port 4gbps Qlogic , and unfortunately I do not have the info on the Fiber Fabric. From what *they* Say, the switches are not being overloaded or the bottleneck. – coderwhiz May 06 '11 at 18:47

1 Answers1

4

Without more details this is tough to answer. 2 things that come to mind are:

  1. the 3par is rebalancing the lun
  2. someone else is using the san

2 is the most likely scenario. Remember that the 3par, by default, shares all disks across all luns (much like the HP EVA). This means that one application or server could bring your performance thru the floor.

Jim B
  • 24,081
  • 4
  • 36
  • 60
  • OK. If we disregard though other users on the device, does the ratio of Random IOPS to sequential hold up? We are at least getting over 2000 IOPS now with Random IO and 5 threads , but still at around 1MB/s throughput. Is this normal at 4KB sizes or does this seem low? – coderwhiz May 06 '11 at 18:48
  • you should always expect the random read and write prformance to be much much worse than sequential becasue you are effectively nullifyig any caching that might be helping. Additionally the second you intrduce any high volume random IO system (eg exchange, SQL server etc) any cached based performance gets pummeled. Without data on what the SAN is capable (number of disk, fiber topology etc) of you have no idea what type of performance to expect. It sounds like the only thing you need to worry about is if your servers are provided enough throughput and if not complain to the san team – Jim B May 10 '11 at 19:38
  • 1
    So oddly, our Service Provider "by accident" was over-subscribing our SAN. Of course its up to the customer to run through myriad benchmarks to figure out where the bottlenecks are and tell them, "Hey , your shits Broken , fix it!" – coderwhiz Aug 24 '12 at 04:08