1 |
On Fri, Aug 26, 2022 at 10:09 AM Mark Knecht <markknecht@×××××.com> wrote: |
2 |
> |
3 |
> Obviously you can do what you are most comfortable with but to me a NAS machine with a bunch of external drives does not sound very reliable. |
4 |
> |
5 |
|
6 |
I would have thought the same, but messing around with LizardFS I've |
7 |
found that the USB3 hard drives never disconnect from their Pi4 hosts. |
8 |
I've had more issues with LSI HBAs dying. Of course I have host-level |
9 |
redundancy so if one Pi4 flakes out I can just reboot it with zero |
10 |
downtime - the master server is on an amd64 container. I only have |
11 |
about 2 drives per Pi right now as well - at this point I'd probably |
12 |
add more drives per host but I wanted to get out to 5-6 hosts first so |
13 |
that I get better performance especially during rebuilds. Gigabit |
14 |
networking is definitely a bottleneck, but with all the chunkservers |
15 |
on one switch they each get gigabit full duplex to all the others so |
16 |
rebuilds are still reasonably fast. To go with 10GbE you'd need |
17 |
hardware with better IO than a Pi4 I'd think, but the main bottleneck |
18 |
on the Pi4 I'm having is with encryption which hits the CPU. I am |
19 |
using dm-crypt for this which I think is hardware-optimized. I will |
20 |
say that zfs encryption is definitely not hardware-optimized and |
21 |
really gets CPU-bound, so I'm running zfs on top of dm-crypt. I |
22 |
should probably consider if dm-integrity makes more sense than zfs in |
23 |
this application. |
24 |
|
25 |
-- |
26 |
Rich |