Hyper-V Cluster network not reachable between nodes

Question

Microsoft recommends a Hyper-V Cluster has its own dedicated cluster network that will be:

Used for inter-node cluster communication such as the cluster heartbeat and Cluster Shared Volumes (CSV) redirection.

With help from the following guide on creating a 2-Node Hyper-converged Cluster with Windows Server 2016, I used the following PowerShell commands to create the cluster network on the first node:

# Create Virtual Machine Switch by merging the management adapters and disabling management OS
New-VMSwitch -Name SW-1G -NetAdapterName Management-1, Management-2 -EnableEmbeddedTeaming $True -AllowManagementOS $False
# Add Virtual Machine Network Adaptors for cluster based on merged switch
Add-VMNetworkAdapter -SwitchName SW-1G -ManagementOS -Name Cluster-100
# Set Virtual Machine Adapter VLAN for Cluster adapter
Set-VMNetworkAdapterVLAN -ManagementOS -VMNetworkAdapterName Cluster-100 -Access -VlanId 100
# Set IP address for cluster adapter
New-NetIPAddress -InterfaceAlias "vEthernet (Cluster-100)" -IPAddress 10.10.100.11 -PrefixLength 24 -Type Unicast | Out-Null
# Disable DNS registration of Cluster network adapter  
Set-DNSClient -InterfaceAlias *Cluster* -RegisterThisConnectionsAddress $False

This then creates the following entry in ipconfig /all

Ethernet adapter vEthernet (Cluster-100):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2
   Physical Address. . . . . . . . . : 00-15-5D-00-6C-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::29bd:4937:2dc1:8f8%10(Preferred)
   IPv4 Address. . . . . . . . . . . : 10.10.100.11(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 503321949
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-CB-79-8B-FC-AA-14-ED-C6-BA
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

The same script is run again on the second node, changing the IP address to 10.10.100.13 which creates the following entry in ipconfig /all:

Ethernet adapter vEthernet (Cluster-100):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2
   Physical Address. . . . . . . . . : 00-15-5D-00-68-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::c823:10d5:66f5:b8bb%9(Preferred)
   IPv4 Address. . . . . . . . . . . : 10.10.100.13(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 503321949
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-CB-79-25-FC-AA-14-ED-C6-A2
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

After running the Test-Cluster node1, node2 command, the report gives the following error within the "Validate Network Communication" section:

Network interfaces node1.lab.com - vEthernet (Cluster-100) and node2.lab.com - vEthernet (Cluster-100) are on the same cluster network, yet address 10.10.100.13 is not reachable from 10.10.100.11 using UDP on port 3343.

Network interfaces node2.lab.com - vEthernet (Cluster-100) and node1.lab.com - vEthernet (Cluster-100) are on the same cluster network, yet address 10.10.100.11 is not reachable from 10.10.100.13 using UDP on port 3343.

Since the cluster network is supposed to allow inter-node communication, it appears the network has not been configured properly. Any ideas what I could be doing wrong?

Are those adapters connected to the same Layer 2 network? Can you communicate between the nodes using any other means/method, such as ping, RPC, SMB, etc.? — joeqwerty, Dec 10 '16 at 05:28
Nodes can ping themselves e.g. On node 1 `ping 10.10.100.11` works but pinging to another node doesn't work e.g. `ping 10.10.100.13`. — ADarkDivideGem, Dec 10 '16 at 05:33
How are they connected? Can you access any of the hidden shares on one node from the other node and vice versa? — joeqwerty, Dec 10 '16 at 05:36
They are connected via a second management network which works fine and is probably why the `Test-Cluster` report also says "Multiple communication paths were detected between each pair of nodes." The only difference between the management and cluster network is the cluster network uses a VlanId and doesn't have a Default Gateway yet the network still passes through the same physical switch/router. — ADarkDivideGem, Dec 10 '16 at 05:44
Looking a little closer at your script, you've configured your vSwitch with a VLAN ID. Have you configured the physical switch ports that these hosts are connected to as trunk ports and allowed the appropriate VLAN's on those ports? — joeqwerty, Dec 10 '16 at 05:49
That's a no, after trunking the ports and enabling 802.1Q VLAN and adding the VLAN ID 100 to the required ports and specifying the untagged ports it worked. Thanks for you helpful advice, I haven't worked much with VLANs before. You can post that comment as an answer and I will accept it. — ADarkDivideGem, Dec 10 '16 at 06:11

score 2 · Accepted Answer · answered Dec 10 '16 at 15:19

Looking a little closer at your script, you've configured your vSwitch with a VLAN ID. Have you configured the physical switch ports that these hosts are connected to as trunk ports and allowed the appropriate VLAN's on those ports? In order to carry traffic for that VLAN the physical switch ports need to configured as trunk ports to carry that VLAN.

Hyper-V Cluster network not reachable between nodes

1 Answers1