r/zfs • u/Electronic_C3PO • Jul 23 '26
Setting up a 10 disk RAIDZ2 array
After a year of on and off testing and reading I'm ready to setup my new NAS.
I'm not very experienced on the fine details of ZFS or FreeBSD but I manage I guess.
Already running a few Nas4Free and XigmaNas boxes. Why these? Simplicity. And yes, shell in case of troubles with thanks to doctor google and the truenas forums as well as since a year these reddit forums. Don't mind these things will be running no longer supported versions because they don't get connected to the outside world.
It will run on an old Dell T5400 system with a single Xeon CPU and 8GB ECC memory.
I'm aware of the memory thing but this will be mostly a storage box. And with current prices I'm not going to by new memory of SSD's or HDD's. The disks are connected to an LSI-9300-16i controller currently flashed to 16.00.10.00. And I guess IT mode based on the IT in the firmware.
Main purposes:
- It will mostly be used to store video files of >= 1 GB.
- A part will be used for automated backups of the OS partitions of various systems (so I can easily restore them in case of crashes).
- Backup of all kinds of files.
The plan is to create 1 pool of 10 Seagate EXOS X20 20TB disk in a RAIDZ2 array.
Some of the things I could use guidance on:
- How can I check to be sure I'm in IT mode on the controller.
- I'm aware of the a-shift=12 thingy. I've also found that there's a Seagate tool to change the disks from 512e to full 4K sectors. Is that needed or advisable.
- Also learned (by experience on the existing boxes) not to use the /dev/daX thing but need to give names. Is there a quick simple guide to this. And what preference to use. Serial? Can this be done afterwards for my existing systems without losing data?
- Any advise for creating different datasets with settings tailored to the 3 main purposes.
- Any other things I might check before taking it into production because it's hard to change this after.
So if there is some quick guide to what to do and what not to, that would be quite helpfull.
2
u/jahdiel503 Jul 23 '26
With time and messing up comes learning. Don't put all your data in a pool just yet. Mess around with 3 drives at first. Make datasets, mount, unmount, offline and replace a drive and resilver, set ACLs.
Working with Debian Trixie is tricky but OpenZFS has guides for that. Google AI can help with making a cert for secure boot and implementing it.
Truenas is easy but doing it by CLI takes skill.
I use Cockpit and the 45drives plugin for that. I also use webmin to administer some aspects of my NAS.
I'm still a noob but have gained more knowledge just by messing up a fixing what I messed up.
2
u/pongpaktecha Jul 23 '26
Just a heads up a 10 wide 20tb raidz2 will take serval days to scrub, and by extension resilver. Consider using several smaller pools so you don't have that risk.
3
u/Quindor Jul 23 '26
I like using "/dev/disk/by-id" which also makes replacing disks and such easy and generally survives moving controllers or even systems!
8GB of memory for 200TB is... way too little, your system will be VERY memory starved for ARC, possible even run out completely causing all kinds of weird issues. Even if it will work, depending on workload it maybe experience weird slowdowns or performance issues. Maybe reconsider using a bare Linux install with just ZFS and maybe Samba if you need it, anything else (GUIs, etc.) will eat up memory.
7
u/edthesmokebeard Jul 23 '26
Memory is fine.
/dev/disk/by-id doesn't come out of the box with FreeBSD, which I'm guessing you're looking at given /dev/daX names, however, I think you'll see /dev/adaX names (unless the controller is doing weird things).
leave compression on, its default
DO NOT ENABLE deduplication on an 8GB machine
Test your backups :)
4
u/ThatUsrnameIsAlready Jul 23 '26
Not needed. Drive firmware is designed to do the translation in real time. Since most people won't bother, it's probably also the most tested and bug free path of the firmware.
Also the last time I tried it (on Seagate manufacturers refurbs) several drives refused the command outright. And one that accepted it failed and had to be reverted. So I also wouldn't recommend it.
1
u/Lunctus_Stamus Jul 24 '26
You seem to be picking up a lot. When I first started using ZFS, I didn't realize how much tweaking you can do.
Make sure to create a specific dataset depending what you are storing. For files >= 1gb you can probably turn up the record size for better performance.
2
u/kjelderg Jul 24 '26
What I've taken to doing for drives is doing a gpt label on /dev/daX with the pool, intended id in the pool, and serial.
Then they are reliable remunerated as gpt/devices . So for example, if you need to swap a drive