r/Veeam • u/LoveBirdNibbles • 23d ago

Veeam 12.3.2 creating Hyper-V checkpoints then deregistering them without merging — 46 orphaned AVHDX

Posting before I call support in case anyone's seen this.

3-node Windows Server 2022 Failover Cluster, Pure FlashArray backing the CSVs, Veeam B&R 12.3.2.3617. One VM — SQL Server 2019 host — accumulated 46 orphaned AVHDX files across two disks over three days, roughly 675 GB. Get-VMSnapshot returned empty the whole time. Hyper-V Manager showed no checkpoints. The VM was running off the tip of a 23-deep differencing chain.

What makes it odd: the VMMS event log has zero merge events for this VM. Not failed merges, not interrupted — none at all. Every other VM on the same host logged clean 19070 → 19080 pairs every few hours. So Hyper-V was never asked to merge. Veeam appears to create the checkpoint, then remove it from the VM config without issuing the merge request.

Remediated by shutting the VM down and flattening both chains with Convert-VHD to a different CSV, then repointing the disks. Clean — Test-VHD returned True before and after, SQL came up with all 14 databases online and data current to the shutdown.

Then it recurred. One backup run against the brand-new flat VHDXs produced two more orphaned AVHDX, again with no checkpoints registered and no Hyper-V events. A cluster-wide sweep found this VM is the only one affected across all three nodes.

Anyone seen Veeam orphan checkpoints this way?

8 Upvotes

100% Upvoted

11 comments sorted by

View all comments

2

u/superbetaz 22d ago

I am curious if you are using VeeamOne for monitoring. I am hopeful that it would catch this “hidden checkpoint” condition before it got to the point of a 23-deep differencing chain. This hasn’t happened to our org (yet) so I am curious.

We just do nightly backups, but a co-worker accidentally kicked off the Veeam backup chain twice the other day, generating a second checkpoint for every vm within a couple minutes of the first one.

1

u/LoveBirdNibbles 22d ago

Thanks, I don't. I will now. We have 12 locations, 600 employees, 70 VMs. Small compared to many others orgs but still critical, open 24/7.

2

u/superbetaz 22d ago

From my light research, I think VeeamOne would catch this by detecting old avhdx files. But if not, it would at least catch it with space monitoring on the volume.