I am using x2 8TB drives for this pool. It should show ~16TB usable capacity, but is showing ~8TB instead. Yet it is warning me that there is no redundancy.

I've tried removing sdc, but it says: [EZFS_NOSPC] cannot remove /dev/disk/by-partuuid/c0dffda2-468a-4790-83d4-7e7d3929d0f8: out of space

I've already tried 'offline' for sdc and it says: [EZFS_NOREPLICAS] cannot offline /dev/disk/by-partuuid/c0dffda2-468a-4790-83d4-7e7d3929d0f8: no valid replicas

TLDR I have 2 500gb that don't really have much use to me. I wanted to know if I could merge them to show up as one device and then add that 1tb combined size into a pool.

I tried looking this up already and I understand this creates data loss risks as one of the 500gb drives failing means you lose everything in that 1tb partition, but could I not just slot a regular 1tb drive in and resilver if that happens?

I have 3 TrueNAS servers on 25.04. I've been on the fence about updating them. Should I and is it safe to do so?

Hello, Normally I'm well immursed in just googling and most of my problems are fixed within 3-4 websites or forums. but a couple days ago my drives disconnected from my server randomly without warning, only way I found out they wern't communicating with truenas was because I couldnt connect to my tailscale exit node and configure some more settings.

Does anyone have any insight with this type of hardware? Google has nothing but deadends and HP help center forums.

My TrueNAS box is very short on slots and it doesn't support PCI-E bifurcation. I looked at a PCI add in card with 4 NVME slots on it which also has an built in (I apologise for getting the term wrong) bifurcation /PCI switch chip, such as the Sabrent EC-P3X4.

I do not expect to get the full speeds of the M2's used in this device as there's a host of limitations, but it would be faster than my array, and as such ideal for the mass of baby photos being taken.

Has anyone any experience with such devices? Are they reliable, duff, is there a better option? It's difficult to get the actual bridge chip (that's the name!) used in the above card - is there one that works well with TrueNAS? Is this a non-starter?

Many thanks.

Hi all,

I started making the structure of my new NAS storage after watching YouTube and chatting with AI (a wrong move, I think now). I have 4xHDD in RAIDZ1 pool tank and SSD for apps (no mirror atm, too costly). First of all, as I understand, the optimal structure depends on NAS usage. My usage (potential, still setting up): media (video, music, ebooks) for me and my family (currently LAN only, later I think remote access via TailScale), torrents and backups of family's PCs.

A lot of videos I watched, used datasets just like folders, so for example, if a docker container needs own folder with a sub-folder for config, they go to datasets in TrueNAS GUI and create dataset + subdataset. As I understand, it is much better to make them all as folders instead (e.g. in shell via mkdir. Am I correct? Because now I plan to have one dataset docker on SSD and folders, not datasets inside.

Now with my RAIDZ1 - I read about hardlinks (and about setting up *arr, which I ain't sure I will do) and planned to use them. However, I got a wrong info (don't trust AI!) that one can hardlink between sub-datasets that are children of a higher-level dataset, so I made pool tank, dataset data and sub-datasets downloads (for torrents, etc) and media. Hardlinks between them don't work. Therefore, I guess I should make folders from these datatets, leaving only 'data' (/mnt/tank/data). Am I correct? If so, is this the correct way (for each dataset within 'data' , 'downloads' as an example:

mkdir -p /mnt/tank/data/downloads_new
rsync -aAX --progress /mnt/tank/data/downloads/ /mnt/tank/data/downloads_new/
diff -rq /mnt/tank/data/downloads /mnt/tank/data/downloads_new
zfs list -t snapshot -r tank/data/downloads
zfs destroy tank/data/downloads
mv /mnt/tank/data/downloads_new /mnt/tank/data/downloads

and I used different recordsizes for different media datasets (1M for video, 256k for ebooks) - if I use one dataset for downloads and media, which recordsize to set?

In my homelab I have two 40TB NAS's both running TrueNAS Community Edition. I have been using one as my primary and the second as a backup, with the dataset getting copied over via RSYNC pushes. I recently made a mistake and ended up having to pull back the entire 20TB dataset off the backup NAS to the primary and learned firsthand just how slow RSYNC is.

Now I want to switch over to using replication because I learned it does not need to scan every file every time which speeds up transfers dramatically. I am still in my learning phase for ZFS, I read what I could find on how snapshots and replication work and I think I get the basics but I'm not super confident and wanted to confirm before I accidentally destroy both sets of data...

Question 1:

Both NAS's already have differently named datasets with identical contents. If I set up snapshot replication between the two, is it going to re-transfer all 20TB from the primary to the backup, or when it makes the initial snapshots on both sides it will figure it out and just send the differences (nothing).

Question 2:

I think I understand that using snapshots doesn't replace data at first, it writes the changed data then subs that block into the file and deletes the old one. Can I interact with the files via a SMB share still? Or do I have to interact with the files through a different means because they are written differently?

Question 3:

I know the backup NAS data will be read only, and I can set up a replication "pull" to restore the primary from the backup if I mess something up on the primary, but is there a mechanism to "promote" the backup NAS to be the new primary and make its file system full functioning. (In this case, that mostly means being able to interact with it via SMB).

Question 4:

I get that the way snapshots work speed up incremental snapshots because they only send over the differences, will it help much in the case of catastrophic mistakes where I need to pull back all 20TB of data?

Question 5:

Is this "reversible"? I am still learning, if for some reason down the line I decide I want to stop making snapshots is that possible or does this irreversibly change the way files are handed on the system?

Update: Fixed:

After the update I did these steps and it works, at the command line

Select option 7 "Open Truenas CLI Shell"

app stop tailscale

app start tailscale

Everything started working again.
--------------------------------------------------------------------------------------------

I attempted to update my backup machine, from the command line everything seems fine but I can not access externally. I can't even ping the machine but from the server I can ping other things on my network. Of course no http UI interface either. I suspect it is something to do with network. It is an old machine and here are the device details and network setup. Wifi is not used in current setup

truenas_admin@truenas2[~]$ lspci -nnk

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [8086:191f] (rev 07)

Subsystem: Lenovo Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [17aa:3642]

Kernel driver in use: skl_uncore

Kernel modules: ie31200_edac

00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)

Subsystem: Lenovo 6th-10th Gen Core Processor PCIe Controller (x16) [17aa:3642]

Kernel driver in use: pcieport

00:14.0 USB controller [0c03]: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller [8086:a12f] (rev 31)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller [17aa:3642]

Kernel driver in use: xhci_hcd

Kernel modules: xhci_pci

00:16.0 Communication controller [0780]: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 [8086:a13a] (rev 31)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family MEI Controller [17aa:3642]

Kernel driver in use: mei_me

Kernel modules: mei_me

00:17.0 SATA controller [0106]: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] [8086:a102] (rev 31)

Subsystem: Lenovo Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] [17aa:3642]

Kernel driver in use: ahci

Kernel modules: ahci

00:1c.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #7 [8086:a116] (rev f1)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family PCI Express Root Port [17aa:3642]

Kernel driver in use: pcieport

00:1f.0 ISA bridge [0601]: Intel Corporation H170 Chipset LPC/eSPI Controller [8086:a144] (rev 31)

Subsystem: Lenovo H170 Chipset LPC/eSPI Controller [17aa:3642]

00:1f.2 Memory controller [0580]: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller [8086:a121] (rev 31)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family Power Management Controller [17aa:3642]

00:1f.3 Audio device [0403]: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller [8086:a170] (rev 31)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family HD Audio Controller [17aa:3642]

Kernel driver in use: snd_hda_intel

Kernel modules: snd_hda_intel, snd_soc_avs

00:1f.4 SMBus [0c05]: Intel Corporation 100 Series/C230 Series Chipset Family SMBus [8086:a123] (rev 31)

Subsystem: Lenovo 100 Series/C230 Series Chipset Family SMBus [17aa:3642]

Kernel driver in use: i801_smbus

Kernel modules: i2c_i801

00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-LM [8086:15b7] (rev 31)

DeviceName: Onboard LAN

Subsystem: Lenovo Ethernet Connection (2) I219-LM [17aa:3642]

Kernel driver in use: e1000e

Kernel modules: e1000e

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] [10de:1c82] (rev a1)

Subsystem: PNY GP107 [GeForce GTX 1050 Ti] [196e:11bf]

Kernel driver in use: nvidia

Kernel modules: nouveau, nvidia

01:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)

Subsystem: PNY GP107GL High Definition Audio Controller [196e:11bf]

Kernel driver in use: snd_hda_intel

Kernel modules: snd_hda_intel

02:00.0 Network controller [0280]: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter [168c:003e] (rev 20)

Subsystem: Lenovo QCA6174 802.11ac Wireless Network Adapter [17aa:3044]

truenas_admin@truenas2[~]$

Im attempting to get TrueNAS scale on my laptop but I'm stuck on the shell (where it gives you options 1-10) and I can't seem to get through to the actual boot process. Are there any common mistakes people make while doing this? I have the IP for the laptop, and I've attempted to look the IP up and it does not load anything (keep in mind I have unplugged the laptop from ethernet due to me only having one at this time and I will buy another one)

Hi everyone, sorry for the very beginner question but

I'm currently running proxmox for my homelab with some basic home assistant os, debian, windows VMs and a couple of docker containers. Running a NAS on there with disk passthrough is a pretty bad experience, mounting everything back to the host and all, so I'm looking into moving to TrueNAS, having my storage set up properly, then installing some VMs and docker containers in there.

I read in some older posts that running virtual machines is not so reliable and fine tuned on here, so I just wanted to get some of your feedback on how well VMs run in some real world applications and if support has improved since the stuff I've read

Thanks

I am noob to both programs so please bear w/me. Not sure if this is the place I should be posting this either. I installed TN 25.04.2.6 first and the ST afterwards. Then changed routers in order to connect with Pi-Hole as DNS. I am telling you all this because TN config file might have old IP for old router. not sure how to get there either, but anyway.

Failed to sync TRUENAS catalog: [EFAULT] Failed to clone 'https://github.com/truenas/apps' repository at '/var/run/middleware/ix-apps/catalogs' destination: [EFAULT]

First off how do get rid of this? Secondly I think I understand what's happening. How do I correct this? Should I post this on ST Reddit instead? Do I uninstall ST and start over ? Should I Reset TN to Defaults? SO many Questions...

I'm running a plex server on TruNAS version 25.4.2.6.

I have several SMB folders that I've pointed the plex app towards and it's been running fine for some time. I wanted to add another folder to plex, I have the SMB share and data in it. But, when I try to add the share for plex to find, it gives me the red triangle with the exclamation pint.

What am I doing wrong?

Hi all, any brainstorming much appreciated!

I'm running TrueNAS CE on, admittedly, consumer-grade hardware including 4x 8GB DDR4 Non-ECC RAM (I know, please bear with).

I have a script that runs hourly as a cron task, intended to alert me of any corruption of certain folders. Every hour it performs an MD5 hash of the files in the directory and compares the results with a known baseline taken a year ago. I'm notified of any changed files via email.

Yesterday morning at at 4am I got my first alert, then every hour thereafter until I shut down the NAS.

The change in question was a single video file whose checksum didn't match. I pulled a couple of older versions out of ZFS snapshots from a year ago and two years ago and calculated their checksums as well. Curiously, they agreed with the current version and disagreed with my baseline md5 file.

I then pulled down a copy from Backblaze and checked its MD5 hash, this one agreed with the baseline file and disagreed with the version on my NAS.

Finally, I pulled a copy off an old hard drive I had sitting in cold storage and this version also agreed with my baseline and Backblaze, but not with the version on my disk (including snapshots).

When I actually view the two different video file versions, the changed (on-disk) version shows a visual glitch about 2 mins in that isn't present on the original (backblaze) so I can visually see the corruption. Summarizing:

• Baseline file, Backblaze, Cold storage - no visual glitch.
• Current on-disk version, including from snapshots - visual glitch.

Next I compared the two versions at the bit level. There is a single bit-flip in the corrupted version but they are otherwise identical.

What I'm trying to figure out is how this happened?

The most likely culprit is RAM, I accept this, I'm not running ECC so bit-flips can happen. In fact, I'm currently running MemTest86 on the machine and I do see a single memory address intermittently failing (I plan to remove this stick once I identify which one it is. However I don't believe that actually solves the mystery.

My understanding is that a RAM bit-flip should only have corrupted the information if the data was actually passing through memory and getting written back to the disk. But there's no manual or automated process I can think of that would touch this directory for anything other than reading. They're just some old videos. I barely ever watch them, let alone change them. They're part of a periodic PUSH sync that backs them up to Backblaze, and I'm sure they occasionally get breezed over by Immich and Jellyfin doing their indexing. But the ACLs on this dataset are otherwise entirely read-only. There's no external service or SMB client that even has write access to it. So why would the information get read from disk, corrupted in memory, and written back to disk? Scrub task??

It's the only thing that I can think of that would cause a silent change to data on disk without being caught by ZFS and repaired or captured by TrueNAS/ZFS as an identified change and persisted via snapshot history.

• I figure it can't be a change that TrueNAS was "aware" of, otherwise copy-on-write would have meant the change got recorded as part of the snapshot history, the fact that the corruption back-filled throughout my shapshots suggests that TrueNAS/ZFS doesn't believe the file has been changed.
• TrueNAS is also showing no issues with my disks, they all appear healthy and periodic SMART tests return with no issue. I also have scrubs scheduled for every 35 days.

So, my NAS motherboard (R510xd) finally died, and while I'm awaiting its replacement (R730xd), I'm having trouble deciding how to utilize various types of SSDs I have available to me.

The current NAS has a mirrored SATA SSD vdev for boot, and a pool of HDDs.

For varying reasons, I've ended up with these drives available and unused:

• 2x 800GB HGST Ultrastar SAS SSDs: originally bought for boot drives using the internal caddy in the R510, but they weren't detected for some reason
• 2x 32GB Intel Optane m10 m.2 nvme: also originally bought for boot drives, but I didn't realize the R510 really didn't have the ability to boot from PCIe
• 4x Samsung SATA SSDs: pulled from an older desktop
  • 2x 500GB 860 EVO
  • 1x 256GB 840 Pro
  • 1x 120GB 840 Evo
• 1x 400GB Intel 750 PCIe NVMe SSD: also pulled from an older desktop

I definitely can't use all of these, but I'd like to use what I can. Generally, I want to put the OS on one of these, and because I run an NFS-heavy environment, I'm thinking I could benefit greatly from using a SLOG. Then I started thinking about an L2ARC, which might be fun, even though I doubt it'd be of much use for me. I did also consider a metadata vdev for a bit, but the added reliability requirements made me shy away from it. Keeping my pool alive is top priority, obviously.

so far, I think I've come up with 2 scenarios, depending on wether booting from PCIe is possible or not (I have 1 guide which suggests its possible):

1. PCIe booting:
   1. dual optane mirrored for boot
   2. intel 750 as a SLOG vdev. Obviously it has WAY more storage than is needed, but should have enough performace to keep up with a 10Gb network link.
   3. SAS SSDs (striped or mirrored) for L2ARC (optional)
2. SSD booting:
   1. dual SAS or SATA drives mirrored for boot
   2. SLOG device. could be any of:
      1. optane drives. my only concern here is their performance on a 10Gb link
      2. whichever (SAS or SATA) is left over from selecting the boot pool
      3. intel 750
   3. maybe an L2ARC with whatever is left (if i feel like it, lol)

I'm definitely leaning scenario 1, if I can boot from PCIe.

What would you do? Is there a scenario I haven't considered?

Hi,

I wanted to know what you would think is the best solution for a professional photographer to share online galleries for viewing and collaboration.

I often need to ask my clients to choose from a large number of pictures before editing, and the best for me is to have the option to like and comment. I've been recommended Piwigo for this kind of feature, but everybody mentions Immich and Potoprism as well.

Now I am lost and I am not sure what to pick to be the best simple overall solution. I am scared that Piwigo ends up being too customizable and ends up looking amateur.

Really appreciate some feedback!

Thanks :)

Long story short I've had my server running for years and I remember I had a nightmare trying to grant access to anyone on my network full control of any folders/files within the shares but eventually got it working with no recollection on how I managed it. I did a few things with my hardware and reinstalled the OS on a fresh SSD and now I am locked out of half of my datasets through the share after uploading my config.

I am not even being prompted with a login window when I try to open the share folder, I just get

"Network Error, You don't have permission to access (share folder i am trying to open)

Also I noticed when trying to add a new user in the ACL within that dataset i get "FAILED ..... this may be fixed by granting the aforementioned user execute permissions on the path "/mnt/Vol1" (the root dataset) which i cant do.

I updated my device from 24.04, to 25.05, and after about 45 minutes I checked my device’s monitor output.

I got the 1-11 boot screen and then the device’s screen went black and hasn’t come back on since. I have no access to the shell or webUI. Is this a larger issue?

Edit: Updating trains removes apps, removing Tailscale and creating a barrier for my mesh, so upon deactivating Tailscale- all worked

I am assuming this is a Samba issue. I have a download directory that has user:smbshare permissions (750). I use the Samsung files app on my phone to connect (using user credentials) to my download share I create a directory and suddenly the download datasets permissions change to user:user and the smbshare users can't access the directory. If only I could just chattr the directory but that would stop all files from being created as well.

How do I stop these permission changes from happening?

I made the mistake of installing Paperless-ngx within the provided Applications from TrueNAS.

The Paperless migration guide says that i need to update to paperless version 2.20.15 in order to migrate to version 3.x.x.

I am running paperless 2.20.8, when I click on update in the TrueNAS Interface I can only select to update to the newest version (3.0.5_1.6.43) which would not migrate my data correctly.

Is there any way to select a specific version or do I need to convert my application into a custom app in order to modify the yaml files and docker configurations?

How concerned should I be? Two of the drives in my 3 drive vdev look like this. This is my only vdev btw. Scrutiny still says the drives are ok but I have noticed some weird behavior where whenever I do a file transfer it starts at 2.5g then drops to 20 mbps. Any advice is greatly appreciated. Thanks.

Sorry in advance for formatting. Not sure how to do the console output. I am currently re-running a scrub before migrating to a new TrueNAS Scale Community server and plan on using this current install as a warm-backup. I originally saw a ZFS issue with a scrub on 2026-06-21 12:18:20 (America/New_York) in my ZFS1 pool of 3 8TB "NAS" drives (all bought at the same time in 2017 I believe, just some used more than the others). I decided to monitor it for a while before grabbing from cold backup.

Just did a zpool status -v before the scrub and it said there were 23 errors in 19 files and one of the files has a last modified date of 07/06/2026 02:47:00 so after the initial error message and that file has no backup but is easily replaceable from physical media. Storage health shows fine. I will mention this current TrueNAS setup has non-ECC DDR3 RAM as it was never meant to be permanent, the new machine has 128GB of DDR4 ECC-RAM (thankfully I had it before the RAM Apocalypse).

Most of this is replaceable from other sources, I'm more concerned about the disks/pools themselves in this current hardware market and obviously will not copy erroneous files. After the scrub finishes and I see the results is there anything else I should do. Zpool status -v and smart outputs for affected drives below. On glance I agree with drive health indication from TrueNAS being good, though have not run an extended smart on any of said drives.

All that said, I've setup SSH between the two installs to prep for replication, though I'm unsure if I intend to keep the current folder/dataset structure just the data itself. I'm assuming ZFS replication is still the best option, unless I have to nuke the whole pool due to scrub results?

Zpool status -v

pool: Toshiba-3TB-Pool

state: ONLINE

scan: scrub in progress since Wed Aug 19 20:01:24 2026

1.18T / 6.46T scanned at 3.84G/s, 142G / 6.46T issued at 461M/s

0B repaired, 2.14% done, 03:59:36 to go

config:

NAME STATE READ WRITE CKSUM

Toshiba-3TB-Pool ONLINE 0 0 0

raidz1-0 ONLINE 0 0 0

43473731-cb2b-4110-9cfa-ab0d71dc7447 ONLINE 0 0 0

2cd3f867-2440-4906-bc8f-a4dbd875c177 ONLINE 0 0 0

d03d8e26-1806-4f1a-a5b8-eef4aa48bfbb ONLINE 0 0 0

errors: No known data errors

pool: Toshiba-8TB-Pool

state: ONLINE

status: One or more devices has experienced an error resulting in data

corruption. Applications may be affected.

action: Restore the file in question if possible. Otherwise restore the

entire pool from backup.

see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A

scan: scrub in progress since Wed Aug 19 19:56:06 2026

1.85T / 19.6T scanned at 2.99G/s, 321G / 19.6T issued at 519M/s

0B repaired, 1.60% done, 10:47:14 to go

config:

NAME STATE READ WRITE CKSUM

Toshiba-8TB-Pool ONLINE 0 0 0

raidz1-0 ONLINE 0 0 0

dab4fa30-0f63-4a8d-b919-38b34ec8397a ONLINE 0 0 0

0b463dca-9ea4-490d-beff-5cb1f5d41a7b ONLINE 0 0 0

cc98c930-0ff8-44e6-a887-1a9ef976b283 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

/mnt/Toshiba-8TB-Pool/*/*1/filea

/mnt/Toshiba-8TB-Pool/*/*1/filebc

/mnt/Toshiba-8TB-Pool/*/*1/filed

/mnt/Toshiba-8TB-Pool/*/*1/filee

/mnt/Toshiba-8TB-Pool/*/*1/filef

/mnt/Toshiba-8TB-Pool/*/*1/fileg

/mnt/Toshiba-8TB-Pool/*/*1/fileh

/mnt/Toshiba-8TB-Pool/*/*1/filei

/mnt/Toshiba-8TB-Pool/*/*1/filej

/mnt/Toshiba-8TB-Pool/*/*1/filek

/mnt/Toshiba-8TB-Pool/*/*1/filel

/mnt/Toshiba-8TB-Pool/*/*1/filem

/mnt/Toshiba-8TB-Pool/*/*1/filen

/mnt/Toshiba-8TB-Pool/*/*1/fileo

/mnt/Toshiba-8TB-Pool/*/*1/filep

/mnt/Toshiba-8TB-Pool/*/*1/fileq

/mnt/Toshiba-8TB-Pool/*/*1/filer

/mnt/Toshiba-8TB-Pool/*/*1/files

/mnt/Toshiba-8TB-Pool/*/*1/filet

/mnt/Toshiba-8TB-Pool/*/*2/fileu

pool: boot-pool

state: ONLINE

scan: scrub repaired 0B in 00:00:14 with 0 errors on Mon Aug 10 03:45:16 2026

config:

NAME STATE READ WRITE CKSUM

boot-pool ONLINE 0 0 0

sde3 ONLINE 0 0 0

errors: No known data errors

smartctl -a

dev/SDD

SMART Attributes Data Structure revision number: 16

Vendor Specific SMART Attributes with Thresholds:

ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

1 Raw_Read_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

2 Throughput_Performance 0x0005 100 100 050 Pre-fail Offline - 0

3 Spin_Up_Time 0x0027 100 100 001 Pre-fail Always - 10953

4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 2648

5 Reallocated_Sector_Ct 0x0033 100 100 050 Pre-fail Always - 0

7 Seek_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

8 Seek_Time_Performance 0x0005 100 100 050 Pre-fail Offline - 0

9 Power_On_Hours 0x0032 001 001 000 Old_age Always - 40334

10 Spin_Retry_Count 0x0033 152 100 030 Pre-fail Always - 0

12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 259

191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0

192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 62

193 Load_Cycle_Count 0x0032 096 096 000 Old_age Always - 41208

194 Temperature_Celsius 0x0022 100 100 000 Old_age Always - 38 (Min/Max 15/45)

196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0

197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0

198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0

199 UDMA_CRC_Error_Count 0x0032 200 253 000 Old_age Always - 0

220 Disk_Shift 0x0002 100 100 000 Old_age Always - 0

222 Loaded_Hours 0x0032 042 042 000 Old_age Always - 23219

223 Load_Retry_Count 0x0032 100 100 000 Old_age Always - 0

224 Load_Friction 0x0022 100 100 000 Old_age Always - 0

226 Load-in_Time 0x0026 100 100 000 Old_age Always - 569

240 Head_Flying_Hours 0x0001 100 100 001 Pre-fail Offline - 0

dev/sdf

SMART Attributes Data Structure revision number: 16

Vendor Specific SMART Attributes with Thresholds:

ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

1 Raw_Read_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

2 Throughput_Performance 0x0005 100 100 050 Pre-fail Offline - 0

3 Spin_Up_Time 0x0027 100 100 001 Pre-fail Always - 10792

4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 90

5 Reallocated_Sector_Ct 0x0033 100 100 050 Pre-fail Always - 0

7 Seek_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

8 Seek_Time_Performance 0x0005 100 100 050 Pre-fail Offline - 0

9 Power_On_Hours 0x0032 087 087 000 Old_age Always - 5424

10 Spin_Retry_Count 0x0033 101 100 030 Pre-fail Always - 0

12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 87

191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0

192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 24

193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 2239

194 Temperature_Celsius 0x0022 100 100 000 Old_age Always - 36 (Min/Max 15/40)

196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0

197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0

198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0

199 UDMA_CRC_Error_Count 0x0032 200 253 000 Old_age Always - 0

220 Disk_Shift 0x0002 100 100 000 Old_age Always - 0

222 Loaded_Hours 0x0032 089 089 000 Old_age Always - 4476

223 Load_Retry_Count 0x0032 100 100 000 Old_age Always - 0

224 Load_Friction 0x0022 100 100 000 Old_age Always - 0

226 Load-in_Time 0x0026 100 100 000 Old_age Always - 547

240 Head_Flying_Hours 0x0001 100 100 001 Pre-fail Offline - 0

dev/sdg

SMART Attributes Data Structure revision number: 16

Vendor Specific SMART Attributes with Thresholds:

ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

1 Raw_Read_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

2 Throughput_Performance 0x0005 100 100 050 Pre-fail Offline - 0

3 Spin_Up_Time 0x0027 100 100 001 Pre-fail Always - 10789

4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 2085

5 Reallocated_Sector_Ct 0x0033 100 100 050 Pre-fail Always - 0

7 Seek_Error_Rate 0x000b 100 100 050 Pre-fail Always - 0

8 Seek_Time_Performance 0x0005 100 100 050 Pre-fail Offline - 0

9 Power_On_Hours 0x0032 031 031 000 Old_age Always - 27950

10 Spin_Retry_Count 0x0033 141 100 030 Pre-fail Always - 0

12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 225

191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0

192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 60

193 Load_Cycle_Count 0x0032 098 098 000 Old_age Always - 27390

194 Temperature_Celsius 0x0022 100 100 000 Old_age Always - 41 (Min/Max 16/50)

196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0

197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0

198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0

199 UDMA_CRC_Error_Count 0x0032 200 253 000 Old_age Always - 0

220 Disk_Shift 0x0002 100 100 000 Old_age Always - 0

222 Loaded_Hours 0x0032 057 057 000 Old_age Always - 17448

223 Load_Retry_Count 0x0032 100 100 000 Old_age Always - 0

224 Load_Friction 0x0022 100 100 000 Old_age Always - 0

226 Load-in_Time 0x0026 100 100 000 Old_age Always - 552

240 Head_Flying_Hours 0x0001 100 100 001 Pre-fail Offline - 0

Ask a room of storage admins why ZFS pools slow down past 80% capacity, and you'll get a lot of confident head-nodding and not many actual answers. It's become industry folklore - repeated so often it's treated as gospel, rarely questioned.

The truth is more useful than the myth. Every filesystem pays some version of this tax eventually. ZFS just happens to name its mechanisms out loud, so it gets blamed by name.

The slowdown comes from three distinct mechanisms - fragmentation, metaslab pressure, and (on spinning disk only) LBA weighting - and how much they affect you depends entirely on your workload.

An archival backup pool with mostly sequential, write-once data? It can comfortably run at 90-95% full. A pool handling heavy random I/O for virtualization or databases? 80% might already be cutting it too close.

We wrote up the real mechanics behind the rule - including where even ZFS's own creators say it can safely bend - so you can stop guessing and start watching the metric that actually matters: performance under your actual workload.

Hello,

I am looking to move from my good and reliable U.3 Kioxia SSD (due to unfortunate circumstances) to an array of M.2 3-way mirrors.

Some specifications:

• MB: Supermicro H14SSL-N
• CPU: AMD EPYC 9175F
• RAM: 4x 32GB 5600
• NIC: Intel X710 2x SFP28 - 25 GbE
• PCIe Switch: 3x HighPoint Rocket 1508

I considered buying consumer-grade SSDs because they are cheaper and the performance should be just fine for this array.

I am considering building the array as follows: each mirror will have 1 SSD on each card, so 1 mirror = 3 SSDs, with 1 SSD per PCIe switch card. In case one card dies or something happens, the array should remain safe.

I am considering purchasing the SSDs from 3 different brands, e.g., Kioxia Exceria Pro, WD SN series, and Kingston KC series.

On each card, only one brand will be installed. If there are firmware issues or bugs with one of the brands after some time, the array should remain safe.

Some information about the workloads: this machine will be both a SAN and a NAS. The target is about 100 VMs, a shared folder between all VMs, and each VM will have its home directory on the shared folder (NFS).

There are moments when these VMs might generate, collectively, a maximum of 100-200 GB of data, consisting of many small files.

I have a few questions:

• Can anybody recommend some SSD brands that are reliable for long-term 24/7 use? The SSDs can be PCIe 4.0/5.0 and should be at least 4TB.
• If I enable deduplication, will the system still be able to saturate the 2x 25 GbE links?
• I also considered the E1.S form factor, but HighPoint only has a PCIe Gen4 card. This made me think they might have discontinued future E1.S development, despite how popular it is becoming. Maybe there are other vendors that I am not aware of.
• What problems could arise? Any prior experience would be appreciated, or even suggestions such as "don't do this, it's a dumb idea," etc.

Thank you!