5

For my department, I envision a two-node cluster of identical machines, running S2D.

I want to use Broadcom 57416 10GBASE-T Adapters (example description), because they support RDMA and run at the speed of the free infrastructure provided by my organization.

Microsoft says that I can connect a two-node setup directly.

Can you tell me what type of hardware and cable would be required for "direct-connect"?

Since I am not using my own switches, I would like to have the nodes sync directly if possible.

Louis Waweru
  • 755
  • 1
  • 9
  • 29

2 Answers2

8

1) You don't need any special Ethernet cabling for direct connection since 1 GbE era, everything will auto negotiate.

https://en.wikipedia.org/wiki/Medium-dependent_interface

2) Broadcom NICs you picked up are shit, you'll get a lot of issues with their performance and stability. Go Mellanox CX4LC family, it's waaaay better.

3) 10GBASE-T has higher latency compared to SFP(+) cabling. Don't do -T especially if you want to cross-connect them and you avoiding a switch.

http://www.fiber-optic-cable-sale.com/10g-technology-10gbase-t-technology-vs-sfp-plus-technology.html

4) Two-node S2D is super-fragile, you have no protection against second disk or node fault during "other" node weekly patch/reboot sequence. Go StarWind VSAN (Free?) which is both cheaper and way faster compared to Microsoft "solution".

https://www.starwindsoftware.com/starwind-virtual-san

BaronSamedi1958
  • 13,676
  • 1
  • 21
  • 53
  • 2
    Thank you, for the advice and warnings. And I like the SFP+ idea. – Louis Waweru Mar 20 '19 at 13:48
  • 2
    I get the DC licenses cost-free through my company, and considering what you said in #4, I brought in [nested resiliency](https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/nested-resiliency) into my solution. I can get the Mellanox ConnectX-4 Lx cards and an SFP28 cable. Considering that the Microsoft licensing has zero cost and the two nodes will be directly attached with an SFP28 cable, and I will use some form of nested resiliency, would you still consider S2D more fragile and wasteful than StarWind VSAN (something I have zero experience with)? – Louis Waweru Mar 21 '19 at 05:14
  • 2
    Cost-free doesn't mean your boss found licenses on the street, CFO is cutting support checks still. – BaronSamedi1958 Mar 22 '19 at 17:33
  • 2
    Nested resiliency requires Windows Server 2019 and Microsoft had pulled it out from the GA as it's really broken :( – NISMO1968 Mar 24 '19 at 20:26
6

For Base-T networking, you have to use Cat6a or Cat7 cables. The main difference between them is the distance (up to 55m in Cat6a and 100m in Cat7). However, as already mentioned, SFP+ is much better in case of bandwidth, stability, and latency.

From year to year (since S2D was released in GA) I've tried to configure it for 2 nodes and it always fails. The stability of work is very poor, the failures to tolerate is quite low and of course, the storage spaces itself. Never ever use the parity SS with HDD. Also, it's very painful to restore the storage spaces direct (and storage spaces) if one of the disks fails (you have to deal with not well-done PowerShell commands).

If you are looking to cluster 2 Hyper-V servers, take a look at other SDS solutions such as Starwind vSAN or HPE StoreVirtual. They are much stable in 2 nodes switchless configuration with convenient GUI (supports both storage spaces and hardware RAID)

Strepsils
  • 5,000
  • 10
  • 14
  • Just curious, were you using a witness machine in your 2-node setup? – Louis Waweru Mar 20 '19 at 13:50
  • 3
    if you are talking about the quorum for the cluster it can be either disk witness (replicated iscsi disk), file share witness and azure witness. As per storage witness, you don't require to have storage witness in Starwind (for example) because of Heartbeat failover strategy. – Strepsils Mar 20 '19 at 15:40
  • 3
    always welcome! – Strepsils Mar 20 '19 at 15:42
  • Thank you, I wound using the Azure cloud witness, both of your commentaries are still very much appreciated. – Louis Waweru Jun 25 '21 at 05:46