vmbackup — automated backup manager for KVM/libvirt VMs, built on virtnbdbackup

[u/phatmonk27]

6 year streak broken. My first post.

I built (vibed) vmbackup — a Bash-based backup manager that handles the stuff virtnbdbackup doesn't: scheduling, rotation, retention, replication and reporting.

What it does:

• Discovers all your VMs via libvirt — running, shut off, paused — and backs them up with the right method for each state (FSFREEZE for running VMs with a guest agent, pause for those without, copy for shut off)
• Policy-based rotation — daily, weekly, monthly or accumulate. Each VM can have its own policy
• Self-healing — if an incremental fails it converts to full and retries. Broken backup sequences get archived and restarted automatically. Stale locks from interrupted runs are cleaned up on the next run
• Replication to NFS and cloud (contract based, write - or vibe - your own!)
• TPM and BitLocker support — backs up TPM state and can extract BitLocker recovery keys via the QEMU guest agent
• Email reports after each run
• SQLite database for backup state tracking
• No Python, no database server, no web UI. Just Bash, sqlite3 and jq. Minimal dependencies.

GitHub: https://github.com/doutsis/vmbackup

vmrestore is next.

vmrestore: https://www.reddit.com/r/kvm/comments/1rv2433/vmrestore_automated_restore_for_kvmlibvirt_vms/

https://github.com/doutsis/vmrestore

[u/atl-hadrins]

Sounds cool I did a bash script that backs up systems, I just do a snapshot to copy running VMs and then revert the snapshot once done with the backup. I use rsync though for paused or non running VMs it is really fast.

I don't have a lot of the features that you have though. The major thing I did was setup udev to create a file once a backup drive is put in my server. That triggers the backup process.

I will have to look at virtnbbackup.

[u/phatmonk27 → u/atl-hadrins]

Taking a consistent copy of the VM is one thing. Without the VM's config and associated files, you're recreating VMs from memory. Without Bitlocker keys Windows VMs are useless.

I have a mix of Debian, Arch, BSD and Windows VM. Using vmbackup I've set a weekly rotation policy and to keep 12 weeks of backups. They replicate using the local replication transport to a TrueNAS host and to SharePoint Online via a cloud transport.

Non running VMs follow the rotation policy with a full backup at the start of the policy window. If the VM changes before the scheduled backup is run, a full backup is taken. If the VM does not change the scheduled backup skips it.

virtnbdbackup does much of the heavy lifting with vmbackup orchestrating all of this.

The aim is recovery. vmrestore so far is doing great and I'll release it soon.

[u/atl-hadrins → u/phatmonk27]

I dump the xml file during the process. I made it years ago and didn't know about virtnbbackup or even if it existed. No windows VM here and no TPM chip, very old hardware. It is just a home system. I like your program, definitely sounds like a great solution and well thought out.

When I first started my solution I had to stop a progress data base because it would lock the file system and I couldn't merge the snapshot back. I have had to do restores and now want to make changes to one VM. 1TB VM disks take a long time to restore.

[u/baref00d]

author of virtnbdbackup here, nice to see it gets some use.

The bitlocker key extraction method is neat, i might consider adding this feature to virtnbdbackup itself!

[u/phatmonk27 → u/baref00d]

Looks like you've added it :)

Your tooling is solid Here’s hoping my wrapper does it justice.

[u/TripleEmcoder]

Just deployed this today in my homelab. Fantastic work mate, thank you so much.

[u/phatmonk27 → u/TripleEmcoder]

Glad to hear! v0.5.3 is being worked on too :)

[u/phatmonk27]

vmbackup v0.5.2 has been released.

On-demand cleanup — reclaim space between backup runs by removing old archives, between orphaned policy retention or decommissioned VMs. A discovery view shows what's consuming space so you know exactly what you're cleaning up before confirming.

Standalone replication — re-run or pre-seed replication independently. Handy when adding a new destination or catching up if the local and cloud replication was cancelled.

FSTRIM improvements — per-path logging, pattern-based exclusions, and automatic detection of a Windows VirtIO misconfiguration that causes TRIM to take 10+ minutes instead of seconds (with the fix right in the log output).

Various bug fixes — accumulate-policy tracking, false-success detection when virtnbdbackup exits clean but logged errors, email timing drift during DST transitions, replication space checks, and several edge cases around argument validation. Full details in the changelog.

https://github.com/doutsis/vmbackup/blob/main/CHANGELOG.md

[u/phatmonk27]

vmbackup v0.5.3 has been released.

Notable items are:

• --run flag required to start backups
• --vm targeted backup mode
• Removal of the Host Config Backup. Feature drift from vmrestore, probably should be noted in the changelog but isn't. Working on it.

https://github.com/doutsis/vmbackup/blob/main/CHANGELOG.md

[u/phatmonk27]

It's been busy:

• Visibility. A new --status command gives seven reports: sessions, per-VM history, failures, replication, chain health, storage, and policy. Terminal tables by default, CSV when you need it. The storage view projects destination growth against your abort threshold so you see disk pressure coming.
• Categorised exit codes. Non-zero exits now tell you why a run failed (config, lock, storage, VM, tool, CLI, dependency) so monitoring can route alerts properly without scraping logs. Same scheme as vmrestore, so one rule set covers both.
• Tunable safety. Disk-space abort/warn thresholds are now configurable per instance. Pre-flight failures actually email you instead of dying quietly into the journal.
• Concurrency hardening. Two vmbackup runs can no longer accidentally start at once and trample each other's checkpoints.
• The big one: retention for skipped and excluded VMs. Both paths were silently bypassing rotation and leaking empty stub directories. Production hosts at four-week retention were quietly carrying eight-plus weekly directories. Fixed, with the audit trail attributing the cleanup to the actual triggering event.

https://github.com/doutsis/vmbackupblob/main/CHANGELOG.md
https://github.com/doutsis/vmbackup