0

using Nexenta or Opensolaris...on server grade h/w: SAS disks + 1GB layer3 switch:

CIFS would only do 10 MB/s. Adding more servers would bump this to 10 MB/sec per server?

dont know about NFS performance on windows.

what is the best way to provision several Tera bytes of storage for Windows server?

looking for I/O speeds like 80 to 100 MB/sec. Sequential I/O

perf tuning ideas: bonding, NIC teaming, 6G SAS?

commercial iscsi SAN from Dell: equallogic, HP: lefthand/saniq...are they any better?

thanks in advance for your thoughts....

JMS77
  • 1,275
  • 5
  • 27
  • 45

2 Answers2

4

If the storage consumers are going to be other Windows servers, iSCSi is a very efficient way to do it. OpenSolaris can do that. Personally I wouldn't trust NFS performance on Windows, others may disagree.

If the storage consumers are actually workstations, CIFS is about the only way to go. In that case you could go for a two-server model, with the storage server providing iSCSI and a second Windows server providing the CIFS serving.

As for efficiency of iSCSI, that greatly depends on what you'll be doing with it. When I last looked at OpenSolaris for just this kind of thing, it seemed to assume the standard transfer size was pretty large. The I/O subsystem in OpenSolaris needed to be tuned to an extent that I couldn't figure out, so small/medium random I/O operations really performed poorly on it (more details here, plus comments).

In my experience OpenFiler performed significantly better for the sort of random I/O I needed for iSCSI. If your AD Domain is small enough, the CIFS engine on that is very good.

Branching into the paid-for storage heads, the LeftHand stuff from HP is really nice. IIRC the latest releases of those have 10G Ethernet as an option, which reduces one of the major bottlenecks I discovered in my own testing. Sequential reads on the storage substystem I had were pushing the 3G SAS limit, but on a single GigE connection that doesn't matter. We didn't go LeftHand because were trying to keep our cost-per-GB on this particular system low.

sysadmin1138
  • 133,124
  • 18
  • 176
  • 300
  • thanks for your response.my Windows workloads are sequential i/o and not random. With occasional spikes in i/o upto 80 MB/sec like once a day. yes my AD domain would be small, windows consumers are couple of server grade machines. are there any perf hits in the two layer model? iscsi -> CIFS? ideas for increasing the bandwidth: using the new 6G SAS, bonding on layer3 GB switch and NIC teaming on storage server? also have you had a chance to see gluster storage platform? www.gluster.com, it looks to me very similar to what HP and Dell iscsi SAN have to offer. – JMS77 Oct 03 '10 at 01:54
  • There are some exceptions, but NIC bonding only delivers 1GB throughput per host/client pair. In your case it sounds like something like OpenFiler would do wonderfully. With the right number of spindles you can leverage 6G SAS, but remember the bonding limit of 1Gb/Pair. Gluster is **very** interesting, but I don't have the nodes to do it right. – sysadmin1138 Oct 03 '10 at 04:08
  • could you elaborate bonding limit 1Gb/pair? what is the min number of spindles needed to leverage 6G SAS? yes Gluster sounds good for claim of doing away with metadata unlike other cluster filesystems.. – JMS77 Oct 03 '10 at 07:04
  • Re: min spindles, that depends entirely on your I/O access patterns and avg transfer size. I know 48 maxed out 3G SAS, and could probably do 6G SAS too. RE: bonding limit, most implementations limit a single tcp connection to a single NIC not the whole family of NICs. Thus 1Gb/pair. To get max bandwidth you need multiple high-rate clients. Alternately you can go 10GbE if you can afford it. – sysadmin1138 Oct 03 '10 at 16:17
1

iSCSI can be faster than CIFS certainly but please consider that it's a block-level system, you can't use it to share data without the use of a cluster-aware file system unlike CIFS/NFS.

Chopper3
  • 101,299
  • 9
  • 108
  • 239
  • thanks for the comment. so this means windows is only limited to SMB/CIFS? how do big datacenter , enterprise guys export storage to Windows servers? with so many storage software available is there a review comparison for these on web? Nexentastor, openfiler,gluster storage etc – JMS77 Oct 03 '10 at 07:55
  • It's not limited to file-level sharing, you just have to use something to allow the block-level sharing to work. Many just use Microsoft's built-in Cluster Services which does an ok job, some use add-ins like Veritas Storage Foundation for Windows - plus there are other options too. It depends how and why you need to share data at the block-level, often file-level is good enough for many situations. – Chopper3 Oct 03 '10 at 10:45
  • sorry i did not fully understand your iscsi comment. how about setting up iscsi target and accessing it on windows using iscsi initiator? – JMS77 Oct 05 '10 at 17:11
  • That'll work, but if you try to get a second machine to look at the same iscsci drive it'll corrupt immediately. – Chopper3 Oct 05 '10 at 17:55