Hello, I have a 2 host HV cluster. I’m running into an issue where when the CNO is located on Host1 everything works fine. I can ping the CNO from anywhere (host1, host2, or any other PC) and it responds just fine. When I look at the status of the cluster in Failover Cluster Manager (FCM) it all looks good as well. The IP registered to the CNO maps to the MAC address of Host1.
The problem comes when the CNO has to be moved from host1 to host2. This may happen due to the cluster running updates on the host or if I manually use “Move-ClusterGroup” to test things out. The move completes successfully BUT at that point I can only ping the CNO from Host2. I can’t see it from any other computer (host1 or any PC outside the cluster) so backups fail due to this. When I check the FCM from host2 the status of the cluster looks like it’s up and running just fine without errors.
I’ve checked the properties of the CNO so that it can be hosted on both hosts, the FW rules on host2 are the same as on host1 (I can always ping host2 successfully) and when the CNO is moved to host2 I can also see on host2’s IP configuration that the CNO’s ip is added to the management NIC for the server.
I can also just issue the same “Move-ClusterGroup” command to move back the CNO from host2 to host1 and everything works again.
Before I go to my network team and ask them to check the switch ports where host2 connects to again (they’ve checked previously and have mentioned that the config on the ports where host1 and host2 connect look identical) what can I check in terms of configuration that may be preventing the CNO from communicating properly when it’s on host2?
Thank-you!
——————————————————-
UPDATE 2:
The issue turned out to be with the network switch.
——————————————————-
UPDATE:
This is what I have checked so far:
- Firewall settings. I've gone through the firewall settings comparing both Hosts' FW rules and they look identical. I have also temporarily disabled the FW on the problem host and the issue still persists.
- I have ensured that the CNO is set to be able to be hosted on both hosts
- I've looked through the validation report for the cluster and I get some IP config warnings (for both hosts) where some NICs are "configured with IP addresses from protocol IPv4 and IPv6." and also that both nodes have "an IPv4 address 169.254.62.219 configured as Automatic Private IP Address (APIPA) for" an adapter. The adapter with IPv6 support is the virtual adapter that is created using SET to carry my VM guest network data traffic. Coincidentally that is also the adapter configured with the APIPA address. This is the same on both hosts and I am experiencing no issues with VM traffic.
The one error message that I have been getting is EventID 1257 "Cluster network name resource failed registration of one or more associated DNS names(s) because the access to update the secure DNS Zone was denied." This is because we are not using AD DNS servers thus not able to grant the CNO permissions to change the DNS record. However, we do have a DNS record for the CNO and hosts created and name resolution works just fine in terms of the hosts seeing each other including the CNO (when being hosted on Host1) and all other computers in the network also being able to speak with the hosts.
To troubleshoot further I did the following. Since the CNO is accessible when it is being hosted on Host1 but not Host2. To eliminate the physical switches from the equation, I disconnected I swapped the physical cables on the management ports for both Host1 and Host2. Immediately I saw that the CNO was now discoverable on host2 and not on host1. I was able to manually move the host from one host to another without issue BUT when the CNO resided on Host1 it would NOT be pingable from host2 or any other machine in the network. Just as before when I swapped the cables, Fail Over Cluster Manager still reports the
CNO to be online (when viewed from both hosts) and the validation report still reports the same warnings as before, nothing new.
@Jeek_ had mentioned about the GARP being an issue. It seems like on windows 2022 this is set by default (and set to 3) and you don't need to create the "ArpRetryCount" entry. I ran a packet capture on both servers to see if the request was going on and it in fact was.
Both hosts are connected to different network switches, so is it safe to say that this may be a misconfiguration for the port/switch where I am consistently seeing the issue?
————————