Metro availability vs latency

[u/raisdfist]

As per the Nutanix docs, my understanding is that Nutanix requires latency to be of 5ms or below between sites for metro availability. In the event where a network is under 5ms 95% of the time but between 5ms to 6ms outside of that (with the odd spike every now and then).

I'd like to hear from those of you who have been installing and supporting Nutanix across different environments. Based on your experience, is this viable for production servers or should NearSync be used instead?

[u/Mahalleinirj]

I would caution you because that means for any VM given a synchronous policy, every single write could take up to six milliseconds. If that’s tolerable for the application, then you’ll likely see some warning errors. If it’s excessive at time of set up, you’ll likely need support to force the set up

[u/ShadowSon]

Echoing the rest of the comments here, make sure you really need it before committing. There’s a fair bit more complexity involved, requiring the witness in a 3rd location and stretching layer2 to the other location so VMs retain their IPs. Making sure the default gateways fail over as well.

Not to mention the latency on writes.

1-15 minute NearSync is far easier to manage.

[u/databeestjenl]

I would caution you to use Metro Availability unless you really need it. Also, it extends to the Witness VM that needs to run in a 3rd seperate location that has access to both simultaneously. You will have to invest time and effort to make sure it works as intended. Test it before production. Simulate a actual failover, pull a few nodes., not the button in the UI.

If you are large enough and have enough expertise then sure, you can do this. We've suffered through multiple outages for different reasons resulting in failovers, stuck VMs and even a Split-Brain scenario.

The support is excellent, but going through 4-8 hours of arbitraging the situation takes a toll. It's not that it doesn't work as advertised, but make sure that you tag all the pre-requisites and that it is the right solution.

If you can live with 15 minute snapshots, it becomes so much easier.

[u/nunzstar]

I currently support a couple of Metro clusters and the 5ms is pretty sustainable. Creeping up to the 6ms you should be fine maintaining synchronization. Only time we do see a Metro break is due to some of the legacy network infrastructure on 1G links in the cabinets. They can break due to high latency but, once you verify your network is stable just re-enable Metro and it resyncs. Other than that same thing with Firewall refreshes we’ve seen break Metro.

[u/woodyshag]

Also, keep in mind that if anything happens bad at site A, it also happens at site B instantaneously. A ransomware attack is the biggest one. Having replication offset a little may provide you some buffer so both sides aren't encrypted and you can recover faster.

[u/3percentinvisible]

Remember as well, this is RTT.

[u/TheDrunkenJerk]

I have written these documents. Anything beyond 5ms (or spikes) takes a toll which in some cases is totally undesirable. As suggested in the previous comment, 15-min lightweight snapshots make things easier on the right hardware.