Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[Robert David]

Hi Frank,

On Tuesday, June 29, 2021 3:56:49 PM CEST Frank Steinmetzger wrote:
> Hello fellows
>
> This is not really a Gentoo question, but at least my NAS (which this mail
> is about) is running Gentoo. :)
>
> There are some people amongst this esteemed group that know their stuff
> about storage and servers and things, so I thought I might try my luck here.
> I’ve already looked on the Webs, but my question is a wee bit specific and
> I wasn’t able to find the exact answer (yet). And I’m a bit hesitant to ask
> this newbie-ish question in a ZFS expert forum. ;-)
>
> Prologue:
> Due to how records are distributed across blocks in a parity-based ZFS vdev,
> it is recommended to use 2^n data disks. Technically, it is perfectly fine
> to deviate from it, but for performance reasons (mostly space efficiency)
> it is not the recommended way. That’s because the (default) maximum record
> size of 128 k itself is a power of 2 and thus can be distributed evenly on
> all drives. At least that’s my understanding. Is that correct?
>
> So here’s the question:
> If I had three data drives, (c|w)ould I get around that problem by setting a
> record size that is divisible by 3, like 96 k, or even 3 M?

I would not bother with this. 128k is a good default for general usage
and even if you got 3 data disks the actual loss is pointless to think
about (assuming you got 4k disks).


>
>
>
> Here’s the background of my question:
> Said NAS is based on a Mini-ITX case which has only four drive slots (which
> is the most common configuration for a case of this formfactor). I started
> with two 6 TB drives, running in a mirror configuration. One year later
> space was running out and I filled the remaining slots. To maximise
> reliability, I went with RaidZ2.
>
> I reached 80 % usage (which is the recommended maximum for ZFS) and am
> now evaluating my options for the coming years.
> 1) Reduce use of space by re-encoding. My payload is mainly movies, among
>    which are 3 TB of DVDs which can be shrunk by at least ⅔ by re-encoding.
>    → this takes time and computing effort, but is a long-term goal anyway.

I always think about in such cases if I really need such data. In many
cases with clear consideration I find out I may remove half of the data
without any pain. It is like cleaning my home, there are many things
extra and there is missing a space for real valuable things, with disk
data it is the same.

> 2) Replace all drives with bigger ones. There are three counter arguments:
>    • 1000 € for four 10 TB drives (the biggest size available w/o helium)
>    • they are only available with 7200 rpm (more power, noise and heat)
>    • I am left with four perfectly fine 6 TB drives
> 3) Go for 4+2 RaidZ2. This requires a bigger case (with new PSU due to
>    different form factor) and a SATA expansion card b/c the Mobo only has
>    six connectors (I need at least one more for the system drive), costing
>    250 € plus drives.
> 4) Convert to RaidZ1. Gain space of one drive at the cost of resilience. I
>    can live with the latter; the server only runs occasionally and not for
>    very long at a time. *** This option brings me to my question above,
>    because it is easy to achieve and costs no €€€.

In any of my data arrays I have long time migrated off the RAIDZ to the
MIRROR or RAID10. You will find finally that the RAIDZ is slow and not
very flexible. Only think you gain is the extra space in constrained
array spaces. For RAID10 it is much easier to raise the size, just
resilvering to new bigger disks, removing old and expanding. The
resilvering speed is magnitude faster. And anyway much easier to recover
in cases of failure. 

If you really need the additional space, consider adding second jbod
with another disks.

Robert.