Re: [gentoo-user] [OT] SMR drives (WAS: cryptsetup close and device in use when it is not)

[Rich Freeman]

On Fri, Jul 30, 2021 at 11:50 PM Wols Lists <antlists@××××××××××××.uk> wrote:
>
> btw, you're scrubbing over USB? Are you running a raid over USB? Bad
> things are likely to happen ...

So, USB hosts vary in quality I'm sure, but I've been running USB3
drives on lizardfs for a while now with zero issues.

At first I was shucking them and using LSI HBAs.  That was a pain for
a bunch of reasons, and I would have issues probably due to the HBAs
being old or maybe cheap cable issues (and new SAS hardware carries a
hefty price tag).

Then I decided to just try running a drive on USB3 and it worked fine.
This isn't for heavy use, but it basically performs identically to
SATA.  I did the math and for spinning disks you can get 2 drives per
host before the data rate starts to become a concern.  This is for a
distributed filesystem and I'm just using gigabit ethernet, and the
cluster is needed more for capacity than IOPS, so USB3 isn't the
bottleneck anyway.

I have yet to have a USB drive have any sort of issue, or drop a
connection.  And they're running on cheap Pi4s for the most part
(which have two USB3 hosts).  If for some reason a drive or host
dropped the filesystem is redundant at the host level, and it also
gracefully recovers data if a host shows back up, but I have yet to
see that even happen due to a USB issue.  I've had far more issues
when I was trying to use LSI HBAs on RockPro64 SBCs (which have a PCIe
slot - I had to also use a powered riser).

Now, if you want to do something where you're going to be pulling
closer to max bandwidth out of all your disks at once and you have
more than a few disks and you have it on 10GbE or faster, then USB3
could be a bottleneck unless you have a lot of hosts (though even then
adding USB3 hosts to the motherboard might not be any harder than
adding SATA hosts).

-- 
Rich