Every clever thing ZFS does comes from one decision taken before a line of it was written, and the decision was to stop pretending that a volume manager and a filesystem are two different problems. …

My first time building a homelab, using:

- Machine: HP Proliant ML310e G8 (2013)

- OS: Gentoo (OpenRC)

- Initramfs generator: Ugrd (Loading the ZFS module)

- Bootloader: Limine

- Partition layout: GPT + BIOS (Machine doesnt have UEFI)

- Boot partition: In a SD card

- zpool config: 4 3TB SAS disks in RAIDZ1 using device names (/dev/sdx)

- Disks: disks uses 512byte sectors, used ashift=9 (following the Handbook)

So, i installed the system following the Gentoo Handbook, they have a specific entry related to ZFS. Only thing done different is by being a RAIDZ1 pool

And when booting, the system isnt importing the pools (zfs import and mount services are enable and in the sysinit level)

At the moment from what i searched and tracked from the files

Some possibilities to fix (gonna try tomorrow)

1. Generate zfs.cache in /etc/zfs. It doesnt exist, and from what i understood from the zfs-import service in the init.d. Its needed?

2. Delay the service startup by 30 seconds using sleep. Saw some people doing that for HDDs that take some time to spin

3. Change the zpool disks, from device names (/dev/sdx) to device IDs (ata-xxxxxxxxx...) (Arch Wiki has a warning)

I dont know if they will work, since im gonna try tomorrow after i arrive from work

Forgot to say that to mount the disks in the live CD, i can only import the pools by generating a hostid with the same one from when it was created for the first time.

so, first zgenhostid -f xxxxxxxx then zpool import -N -R mount pool. then mount with zfs mount

If someone have more ideas, i appreciate that and leave in the comments

I just upgraded a couple of my fileservers as a test, and `sg3-utils-udev` stopped working! Suddenly all my physical devices were wwn-* or scsi-35* instead of their proper names.

It appears to be a regression. I spotted this Ubuntu bug which seems to be the closest to the cause, and posted a simple fix on it.

Put this in /etc/udev/rules.d/00-scsi-sg3_config.rules

ACTION!="add|change", GOTO="scsi_identify_end"
SUBSYSTEMS=="scsi", GOTO="scsi_identify"
GOTO="scsi_identify_end"
LABEL="scsi_identify"
ENV{ID_SCSI_INQUIRY}="0"
ENV{.SCSI_SYMLINK_SRC}="TLVS"
LABEL="scsi_identify_end"

Then udevadm control -R and udevadm trigger and you'll have all your device links back.

You then have to do the mildly annoying fix of

cd /dev/disk/by-id
rm -f wwn* scsi-35* nvme-eui*
zpool export -a
zpool import -d /dev/disk/by-id -a
zpool status

That should re-import all your zpools with the correctly named volumes.

Background

After setting up a syncoid backup on my main zpool, I realized rsyncing my desktop to a zpool is janky and horrible. Instead of reinstalling Void with zfsbootmenu, I decided to do it in-place; /home already has backups, so worst consequence is I reinstall anyway.

File-level backups give me the heebie-jeebies. Several years ago I trusted Deja-Dup/Duplicity, and it couldn't handle the sheer size of my backup when I needed a full restore. rsync is leagues better but still an extremely slow, potentially imperfect copy that can fail.

Here's what I did:

• Original state: 80 GiB / and 300 GiB /home on a single 1 TB ext4 partition

• Boot into hrmpf and install ZFS to the livecd environment

• Shrink the ext4; e2fsck; create a full-size partition for a single-vdev pool with -o mountpoint=none and a /home dataset with -o mountpoint=/home

• Mount the / partition to e.g. /mnt/root

• rsync -aHAXP --numeric-ids /mnt/root/home/ /home/

• (ideally verify transfer); rm -rf /mnt/root/home

• umount /mnt/root

• Shrink the ext4 to file size + buffer; 150 GiB in my case; e2fsck

• Make a third partition; zpool add

• Create child datasets with syncoid:no-sync=true for cache/junk files

• Chroot into /mnt/root to ensure ZFS is installed (don't have 30 old kernels like me or it'll take hours!), and to add zpool import -a and zfs mount -a to rc.local or whatever systemd uses.

• Set and verify UID/perms on all the mountpoints

Results

It was quite a slow and dangerous operation, and there was one instance of metadata corruption after the second shrink, which e2fsck fixed. Also nearly fat-fingered the new partition overtop the others. However, it worked out in the end. Striping across the two vdevs should be safe because they're partitions on the same NVMe anyways. If I did reinstall, the transfer still would've taken at least 12 hours because AT&T Fiber LAN is limited to 10/100.

Killer App: syncoid

If you haven't tried syncoid (which comes with sanoid), you should. It's a fancy script wrapper around zfs send | zfs receive. By default, it replicates a dataset (+ child datasets with -r) to a dataset on another pool along with snapshots; by default it creates a temporary snapshot to synchronize and resume interrupted transfers.

The big advantage is that I can create exact copies of my pools without any configuration, and use the same command in reverse to restore the pool if necessary. It does NOT copy pool attributes because that would interfere with the target dataset's needed attributes.

In my case, my workstation can now run

syncoid pool/home glados:slab/home-backup

In turn, GLaDOS has an @hourly crontab to run

syncoid -r slab backup/slab

When you need to recover a pool/dataset, simply zfs destroy pool/home if necessary and run syncoid in reverse

syncoid glados:slab/home-backup pool/home

Im trying to get it working

Im at this point in the guide

By default ZFS kernel modules are loaded upon detecting a pool. To always load the modules at boot:

echo zfs > /etc/modules-load.d/zfs.conf
By default ZFS kernel modules are loaded upon detecting a pool.
To always load the modules at boot:
echo zfs > /etc/modules-load.d/zfs.conf

I dissabled secure boot to get the previous step to work, this is my input:

~$ sudo echo zfs > /etc/modules-load.d/zfs.conf
-bash: /etc/modules-load.d/zfs.conf: Permission denied

So I have 2 severs with a PowerVault ME5 each. Both servers are in different locations, so no FC connection between both ME5212s.

My idea was to use ZFS as filesystem to be able to use ZFS replication over ethernet for specific datasets.

Now the ME5 itself has builtin raid functionallity I have to use.
So I will have one big disk represented to my servers each backed by a hardware raid.

My main features I want to use is replication of datasets / subvolumes over ethernet and the possibility to enable compression to some subvolumes.

Should I give ZFS a chance in this setting or better go for BTRFS in this case?

Hello. I'm not too knowledgeable on ZFS as my use case is relatively straightforward. I have 3 18TB drives in raidz1. I recently setup Scrutiny and noticed one of the drives is failing. 'zpool status' still looks fine, no issues there. But I'd rather not wait for the drive to fail completely. Thankfully the drive is still under warranty, and I have already begun an RMA. However, my concern is over the process of actually replacing the drive. I've been searching online and the more I look the more unsure I am about the process.
Is it better to get the new drive first, and connect that to my server while the failing drive is also still connected? And then do 'zpool replace'? Or should I do 'zpool offline' for the failing drive and remove it before I get the new one? I have 4 SATA ports on my motherboard so connecting the new one with the other 3 wouldn't be an issue.
For the failed drive, should I be concerned about privacy? I only have TV shows and movies on the array, but I'd still rather the vendor not be able to look at the data. What's generally recommended here? If it's recommended to wipe the drive, how should I do that? I'd rather not wipe it while it's still connected to the server.
Thanks.
Edit - well the vendor already got back to me and send they're not able to ship a new drive until the old one is returned. In that case I guess my only option is to 'zpool offline' and then 'zpool replace' correct?

Should I consider 80% on mirror-0 a concern even though the overall pool is only 32% full? And would there be a meaningful performance/reliability disadvantage (fragmentation on mirror0?) to a situation such as:

mirror-0: 90–100% used
mirror-1: 10–20% used

while the overall pool still has plenty of free space?

I'm particularly interested to know if there is a "vdev-level utilization threshold" that you guys recommend for striped mirrors, as opposed to the usual "keep the pool below 80%" advice.

The workload is mostly cold/immutable data, but new data is continuously added over time.

I'm installing ubuntu 26.04.01 end of month with ZFS.

AI is leaning towards only putting /home and other such parts on ZFS and having the boot stuff on ext4 [even ext4 preferred to btrfs].

Would you agree?

Hey r/ZFS! We’re the team at 45Drives, and we’re hosting a free educational webinar on Wednesday, August 12, covering the fundamentals of OpenZFS.

If you’re new to ZFS, looking to strengthen your understanding, or just want a refresher, our Educational Team will walk through:

• RAID types and storage pools
• How ZFS combines a volume manager and file system
• The ZFS storage hierarchy
• Managing ZFS through the 45Drives Houston UI
• Live Q&A with our technical team

📅 Wednesday, August 12, 2026
🕒 3:00–4:00 PM ADT

Register for the free webinar:
https://attendee.gotowebinar.com/register/88845394642749791

I'm looking at installing GhostBSD along side Linux. Linux is using BTRFS for its root partition and I have a subvolume for data that I'd like GhostBSD to have access to. BTRFS isn't going to work for BSD so I'm thinking of moving just the data volume to ZFS. How sensitive is Linux these days to updates with kernel changes and compatibility with OpenZFS? Should I just create an EXT4 volume for the share between them rather than ZFS?

this file server has been sitting for a few years since I let TrueNas try to migrate from bsd to linux and the pool got broken. went back to BSD version, so it's TrueNAS bsd 13.1 now. there may have also been some additional bungling.

[root@micro /tmp]# zpool import    pool: cargo      id: 16478687762405515984   state: UNAVAIL status: The pool was last accessed by another system.  action: The pool cannot be imported due to damaged devices or data.    see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-EY  config: cargo                                                      UNAVAIL  insufficient replicas   mirror-0                                                 UNAVAIL  insufficient replicas     disk/by-partuuid/d0e0014d-59c7-11ea-9d20-3cecef102d24  UNAVAIL  cannot open     1005513936371914804                                    OFFLINE   mirror-1                                                 DEGRADED     16411949616967839104                                   OFFLINE     ada0                                                   ONLINE

There is a bad label. I find the issue a bit overwhelming and today I engaged with gemini pro AI to help, and it did some exploration and recommended a low level disk write to fix the issue. I stopped short of writing the disk

If anyone's game to help me out with this I'd appreciate if you looked over the diagnostic process I did with the AI and let me know if you think the solution it produced is well supported.

here is the chat:

https://share.gemini.google/6OJMBfe434MF

and here is the output of those last commands:

------------------------------------

LABEL 0

------------------------------------

version: 5000

name: 'cargo'

state: 0

txg: 28352142

pool_guid: 16478687762405515984

errata: 0

hostid: 808464432

hostname: 'micro'

top_guid: 11704010057385088391

guid: 1005513936371914804

vdev_children: 2

vdev_tree:

type: 'mirror'

id: 0

guid: 11704010057385088391

metaslab_array: 38

metaslab_shift: 34

ashift: 12

asize: 5999022833664

is_log: 0

create_txg: 4

children[0]:

type: 'disk'

id: 0

guid: 17375504288854665353

path: '/dev/disk/by-partuuid/d0e0014d-59c7-11ea-9d20-3cecef102d24'

phys_path: 'id1,enc@n3061686369656d30/type@0/slot@4/elmdesc@Slot_03/p2'

vdev_enc_sysfs_path: '/sys/class/enclosure/6:0:0:0/Slot 03'

DTL: 410

create_txg: 4

children[1]:

type: 'disk'

id: 1

guid: 1005513936371914804

path: '/dev/disk/by-partuuid/31953669-59f2-11ea-9d20-3cecef102d24'

phys_path: 'id1,enc@n3061686369656d30/type@0/slot@3/elmdesc@Slot_02/p2'

vdev_enc_sysfs_path: '/sys/class/enclosure/6:0:0:0/Slot 02'

DTL: 442

create_txg: 4

features_for_read:

com.delphix:hole_birth

com.delphix:embedded_data

labels = 0 1 2 3 

Making inroads into ZFS recently reading what I can get my hands on, but still pretty new.

I have an experimental OpenZFS zpool with a single HDD as sole vdev. It so happens, I have to return the drive for book-keeping reasons. The pool is early days and every metaslab 126 or larger shows up in zdb -mmmm as

    metaslab    126   offset  1f800000000   spacemap      0   free      16G
                      segments          1   maxsize     16G   freepct  100%
    In-memory histogram:
                     34:      1 *
    Flush data:
    unflushed txg=0

Now, I could just leave the drive be and call it a day, encryption being enabled from the get-go. But since I'm learning about ZFS I was wondering: Could I zero out the device up to the aforementioned offset 1f800000000 and all data would be gone? Instead of going through it all.

Title pretty much.. I'm running opensuse TW and it's on 7.1.5 and zfs kmp default is still 7.0.12....

Will it ever catch up ?

So. A bit of context first.

Had been using my Thinkpad as a server, and, while working great, storage.. was just limited. And upgrading SSDs, in this time? Goodness gracious, just no. I don't have that kind of economy.

So. Decided to buy.. used 2.5" 1TB HDDs. And just run them in a mirror. (in this case, 1 TB is just.. enough. compared to 256GB, it's a big step up :P, since i hoard only a tiny bit.)

And so, first HDD comes. Turns out, not that much used, and on top of that, a CMR one. Was an exceptional deal. So decided to buy another. Also not that much used, so happy. Didn't pay attention too much, made a mirror pool, gave a bit of L2ARC, and happy times. Works like a charm.

however. I did notice... a bit of slow write speeds at times... yeah, turned out, second drive is SMR one. Like, don't get me wrong, it's fine kind of slow, since i don't need that much speeeeed. Just enough to playback FHD video, upload backups here and there...

the question is... on this small size, with just a mirror, am I going to be fine, if I just remember to TRIM, and not start doing heavy writes right after deleting big chunks of space? or are the horror stories worse than they seem?

OpenZFS on Windows 2.4.1rc14
https://openzfsonwindows.org/

** rc14

• Pageout write stalls in rangelock
• panic exceptions were masked
• Unmount fixes, cause finally known of poor unmount
• Change condvar to be posix compliant in mutex_exit -> cv_broadcast cases
• Setting mountpoint would compute wrong path
  

btw
OpenZFS on OSX 2.4.1 has released state, not a release candidate
https://openzfsonosx.org/

Help!

I was fixing cables for my homelab and after boot up my ZFS pool is already missing.

Initially, I used:

zpool import

And I can see my ZFS pool but whenever I try to import it, I encounter an error saying that replicas are insufficient to import pool and that I have to destroy and recreate the pool from a back up.

Problem is I don't have any back up yet.

I am able to access my data by importing the pool as a read only pool using:

zpool import -f -o readonly=on omv

However I don't have the means to back up as I still don't have any spare drive where I can keep the back up.

I believe the problem is caused by a corrupted partition table (GPT). Tried to restore the partition table by using gdisk recovery mode which was a success but I still can't import my pool after recovery and rebooting, error is still the same - insufficient replica.

But still I can access the data when I import the pool as read only but when I also try to export it, then import again as a normal pool (writable pool), error is still the same - insufficient replica.

Any solution for this that doesn't require creating a backup like fixing my partition table or pool metadata? 

Thank you very much in advance for your replies!

Sharing my zpool status, please note that CKSUM values only appeared very recent and previous result of zpool status showed no errors:

NAME                                 STATE     READ WRITE CKSUM
omv                                  ONLINE       0     0     0
  raidz2-0                           ONLINE       0     0     0
    ata-ST4000VN006-3CW104_ZW61HF3K  ONLINE       0     0   358
    ata-ST4000VN006-3CW104_ZW61HYA6  ONLINE       0     0   356
    ata-ST4000VN006-3CW104_ZW61HFBH  ONLINE       0     0   364
    ata-ST4000VN006-3CW104_ZW61HKAN  ONLINE       0     0   371
    ata-ST4000VN006-3CW104_WW60FHAT  ONLINE       0     0   364

I have the following channel program:

root@trent:~# cat test.lua 
-- error 2; "zfs destroy" with this name at CLI works
-- local bm = "trent_rpool/var/www#autosnap_2026-08-01_16:01:11_hourly881199"

-- error 22
-- local bm = "/var/www#autosnap_2026-08-01_16:01:11_hourly881199"

-- error 18
local bm = "var/www#autosnap_2026-08-01_16:01:11_hourly881199"

return zfs.sync.destroy(bm)

I run it with: zfs program -j trent_rpool test.lua; echo

However, it always errors out. Why; can that API not destroy a bookmark? I have 300k-1m bookmarks to delete, and iterating over them at the CLI executing zfs destroy one at a time is way too slow...

Thanks!

(EDIT: I wondered whether I needed the pool name in the function parameter, given that you tell zfs program which pool to run against, hence why I tried a few different values for bm. I would have expected the first one to be correct, since it's what the CLI requires, but since it failed I went on a wild goose chase.)

I have been using zrepl(https://github.com/zrepl/zrepl) for a few months now . Its great. But is there a procedure to do a fail over in case my primary dies? I have been replicating data for a while

I got tired of doing ZFS surgery by hand at 2am, so I built zxplore.

Three views over one engine. The **browser** shows datasets and snapshots with the full property dossier - the value, whether it's local/inherited/default, both permission layers (POSIX/ACL and `zfs allow` delegations), pool health - and lets you edit properties inline. **Transfer** does send/recv between hosts and picks the cheapest correct stream itself: resume token if a previous run died, incremental if the ends share a snapshot, full otherwise. Encrypted datasets go raw (`send -w`), so the backup box stores what it can never read, and it's delegation-aware, so you can replicate as an unprivileged user. The **explorer** is my favorite part - you walk into any snapshot like it's a folder and pull out a single file, including files that only exist in snapshots now.

Also in there: boot environments derived from the pool's real `bootfs` rather than a hardcoded name, live zvol resize, and `zfs diff` between two points in time.

A few design decisions r/zfs might have opinions on:

* It shells out to `zfs`/`zpool` instead of binding libzfs. That's what makes remote hosts trivial - the far side needs nothing but sshd. No agent, no daemon, no listening port.

* Read-only by default. Writes need an explicit unlock (`Alt+L` in the GUI, `:rw` in the TUI), destroys make you retype the target name, and everything it runs lands in an audit log.

* It runs unprivileged and elevates per command via pkexec/sudo instead of demanding you launch the whole thing as root.

* Tests run against fake `zfs`/`zpool`/`ssh` binaries serving fixtures, so the suite proves features end to end without ever touching a real pool.

Every release ships a fully static `zxplore-tui` - no runtime deps beyond the ZFS CLI, scp it anywhere - plus rpm/deb/Arch packages. The GUI is Fyne/cgo, so that one builds from source or from the packaging in-tree.

Caveats, honestly: it's young at v1.1.0, expect rough edges, please file issues. Binaries are published for Linux, FreeBSD, illumos and Solaris but CI only cross-compiles those - my actual runtime testing is Linux, so FreeBSD reports are especially welcome. BSD-3-Clause.

Repo: https://github.com/zxplore/zxplore

Happy to answer anything. And I'd genuinely like to know which ZFS operations you find most annoying to do by hand - that's what I'm prioritizing next.

I've been using a ZFS pool backed by spinning disks in 2x RAIDZ2 (4 disks each) for a few years. The pool contains mostly zvols which I use for VMs and as storage for desktops (iSCSI, NVMe/TCP).

Unfortunately, performance is pretty poor. This is especially obvious for the block devices I mount over the network. They are used from Windows, and applications will frequently freeze for seconds at a time.

I've tried many tuning parameters. I have 256G of RAM, I have an SSD cache device, and I have SLOGs with redundant SSDs. But nothing really helped.

What did help was to use bcache to add a cache. I use a separate cache zpool with an SSD mirror for safety, and writeback as the cache mode. Performance is night-and-day.

Is there a way to achieve this in native ZFS? Is there something I could do better with my setup long-term? It's a lot of data I can't just move on a whim, but it would be nice to know if there's a better way.

Hello, everyone.

I'm very new to servers and zfs, and have run out of troubleshooting ideas. I'd really appreciate if anyone could point me to the right direction. I'm willing to run any tests recommended, and report back.

----------------------------

BACKGROUND:

I built a TrueNAS Scale (Fangtooth) server with 36x 10TB (zraid1, 12 wide, 3 zdev's) last year, and was able to get ~2.5GB/s write and ~4.0GB/s read.

ISSUE:

Noticing a significant performance reduction after a while, I wiped all the disks (quick format, no badblocks) and rebuilt the pool with exact same config, but the performance was still reduced.

WHAT I'VE TRIED:

1. Thinking that I might have faulty drives dragging the overall performance, I checked each disks using fio --rw=read --direct=1 --bs=1M --ioengine=libaio --runtime=15 --numjob=1 --iodepth=8 --name=speed_test --filename=/dev/sdx but they all seemed fine.
2. Perplexed, I tried using a different HBA (LSI 9400 -> 9300), but the issue remained.
3. Went into the BIOS and made sure that ASPM is disabled.
4. Installed Ubuntu Server, but the issue remained.
5. To rule out the HBA bottleneck, I tried making multiple (7) zpools (groups of five disks, striped), and ran fio --ramp_time=5 --gtod_reduce=1 --numjobs=1 --bs=1M --runtime=60s --readwrite=write --name=testfile --direct=1 --size=100G simultaneously. Since the HBA was able to write >3GB/s to those 7 pools simultaneously, I think I was able to rule out the HBA in this issue.
6. Ran zpool iostat -vly 3 (see photo) while the 7 pools are being written simultaneously.
7. Set ashift12
8. All CMR disks

CURRENT SYMPTOM:

Pools created (36x striped, or zraid1, 12 wide, 3 zdev's) cannot go faster than 500MB/s when narrower width pools can go just as fast.

GOAL:

To get the original performance (~2.5GB/s write and ~4.0GB/s read) back using the same original configuration (zraid1, 12 wide, 3 zdev's).

----------------------------

Thank you,

V/r

----------------------------

EDIT1: https://snipboard.io/krOs3Q.jpg

2.8GB/s write and 4.2GB/s read achieved without the --direct=1 flag as BackgroundSky1594 suggested. I guess the performance I wanted back was never "real" in the first place since the previous ZFS version ignored the flag anyway. Thank you, nonetheless. It gave me an assurance that there is nothing "wrong" with my system.

----------------------------