1

We have a new NAS with 6xSAS in RAID10, testing OpenSolaris 2009.6 on it.

A local read/write speed is amazing, about 700/350MB/sec. The network is temporary set up a s a single 1Gbit link, without any link aggregation/jumbo frames. I get 80/60MB/sec read/write over NFS.

When sharing a volume via iSCSI I get 60MB/sec write speed however a read speed is very slow, about 5-10MB/sec. What could cause this? Tried Linux/Windows iSCSI initators, result is the same.

Thanks for the ideas.

disserman
  • 1,850
  • 2
  • 17
  • 35

1 Answers1

3

iSCSI performance depends a lot on the quality of your networking equipment. A few considerations:

  • Make sure your server and clients have high quality network cards, preferably with TCP offload, or even hardware iSCSI capability.
  • Make sure your iSCSI traffic is on a wholly separate network. Ideally it would have its own physical switches, but having its own VLAN works as well.
  • Don't cheap out on the switches. Make sure you are using one from a vendor known for performance gear (eg: Cisco, Foundry, HP, etc)
  • Use jumbo frames. They can boost iSCSI performance by up to 50% by reducing the TCP/IP overhead

On my home OpenSolaris NAS I briefly tested iSCSI, and the performance from my Windows-based initiator was terrible till I switched to jumbo frames.

Kamil Kisiel
  • 12,184
  • 7
  • 48
  • 69
  • I understand, thanks. But how could it be the problem is only with iSCSI read? I don't say I get a perfect results with iSCSI write/NFS, but they are pretty close to the theoretical values what you could have without a separate VLAN/jumbo frames etc on 1xGBE. The problem is only with iSCSI read and only on OpenSolaris (tested linux-based OpenFiler 2.3, works nice). I've tried built-in zfs iscsi target (SUNWiscsitgt) as well as COMSTAR variation, result is almost the same. On COMSTAR I've got 15MB/sec – disserman Sep 26 '09 at 19:41
  • finally I've found you were right. After replacing a switch everything works nice. It's strange iSCSI target in OpenSolaris is more "sensitive" to a bad network conditions but it is. – disserman Sep 29 '09 at 21:00