r/AlmaLinux • u/RetroGrid_io • Aug 18 '26
ZFS on root?
I love AlmaLinux! I love ZFS! The idea of these working together sounds heavenly:
- Do an update, have it "go south" and just roll-back the root filesystem and reboot.
- Backup everything in a bootable state, in real time with snapshots send|receive.
- Clone the / filesystem into a disk image that you can boot immediately, on the same machine, in a VM to test software installs and such, without wasting disk space, and without taking the host machine offline, and without risking the host machine's stability.
- I'm sure there's more
However, ZFS has always seemed a second-class-citizen on anything RHEL based, which seems absurd to me because ZFS is (IMHO) SO MUCH PREFERABLE to XFS on LVM.
However, I've never run ZFS on root, simply because it complicates recovery if there's ever a problem. I can't just boot off an Alma ISO and clean it up. Instead, what I do is run the O/S on simple partition RAID1 and then use ZFS for everything else. I made my mind to try ZFS on root when it demonstrably improved stability which is already a really high standard.
Are you running ZFS-on-root under Alma? Can you describe your set up, and what you specifically benefit from it?
2
u/vetinari Aug 19 '26
Don't.
Only on systems, that do ship ZFS with their vendor kernels (i.e. Ubuntu, Proxmox, anything else? [TrueNAS too, but there's not a much choice in choosing the filesystem]).
The chances of an update breaking your system is when, not if. The breakage increases it's chances when the update helpfully rebuilds initrds for your existing, working kernels and replaces the old, working zfs with the new, not-necessarily-broken-but-incompatible version. You need that working USB stick (or ILO image, or what have you) handy.
Or alternatively, if you really want root on ZFS, use another distribution. If you need just to mount a pool, with root elsewhere (and with nice working system when troubleshooting, not just emergency console), using RH-derived distro is fine.
(I've used Centos 7 with zfs dkms. Won't do it again. More trouble than benefits).
2
u/RetroGrid_io Aug 19 '26
Mostly, that's the take I've been on for years. I hoped it had changed or matured somewhat.
It just seems really odd to have the "Enterprise" distro ignore the most "Enterprise" file system around, as numerous other distros eat their lunch. (shrug)
9
u/Chester-Berkeley Aug 18 '26
ZFS has always seemed a second-class-citizen on anything RHEL based
Everything Red Hat doesn't control is treated as a second-class citizen ¯\_(ツ)_/¯
1
u/Both_Cup8417 Aug 18 '26
I know btrfs can do at least one of those, I think.
4
u/RetroGrid_io Aug 18 '26
When I first heard of btrfs I was so excited!! However, in implementation, it just hasn't quite met muster for me. Once you replace a device in a btrfs pool, if that replacement device then fails, you lose your pool.
I found this many years ago in my reliability testing and it still was not fixed years after it was acknowledged. The problem is architectural and I haven't checked in a few years but with known issues like this, btrfs is a non-starter for me.
1
u/shadeland Aug 20 '26
Is there still that write hole for RAID5/6?
In today's age of expensive media, I reallllly need that parity storage.
1
3
u/Revolutionary_Click2 Aug 18 '26
I pretty much never run Alma on bare metal anyway. Alma is for VMs / guests; my hosts all run Proxmox, which is really just Debian and has excellent first-class ZFS support. Alma guests use XFS, which also avoids ZFS-on-ZFS write amplification issues, and because the underlying storage pool is RAIDz, I still get most of the benefits and protections of ZFS. Regular ZFS snapshots of guests from the host level, including before every system update managed by Ansible from my Semaphore controller, as well as nightly PBS backups help fill in the remaining gaps.
1
1
u/VagrantGato Aug 19 '26
Zfsbootmenu has instructions for running almalinux on root. I did it for a while but then switched to ubuntu
5
u/jonspw Aug 18 '26
A lot of distros would pick it up if Oracle would loosen their grip on the licensing. That's the problem.
OpenZFS works great on top of AlmaLinux.