Re: [gentoo-user] External hard drive and idle activity

[Dale]

Rich Freeman wrote:
> On Mon, Aug 3, 2020 at 8:24 AM Dale <rdalek1967@×××××.com> wrote:
>> In the past, I've never seen the drive on the larger files be that slow even toward the end.  Generally, it stays pretty close to 180MBs/sec or so which is what I usually get with PMR drives.
> Yeah, just hard to be certain without ditching the filesystem layer or
> doing some kind of comparison.  The difference in write speed across
> the drive on recent drives I've gotten is more pronounced than I've
> seen on other drives, but these drives tend to be around 12TB.
>
> Definitely the thing to watch out for is a big drop in transfer rate
> once a large number of blocks have been transferred continuously, and
> then performance returns after you let the drive thrash for a while.
> I've seen complaints of zfs rebuilds going from hours/days to
> weeks/months in length, so it isn't just a 50% drop when you're doing
> worst-case access patterns.  On the other hand I hear that mdadm isn't
> so bad, so if the writes are sequential the drive might be better at
> skipping the cache, and maybe zfs just does its rebuild
> non-sequentially (which isn't really ideal anyway).
>
> I haven't really dug into the guts of how zfs metadata works, but with
> btrfs I believe the chunks are basically their own layer, and the
> filesystem can scrub them without really any care about what files are
> stored in them.  That allows them to be easily scrubbed sequentially.
> When I did rebuilds on btrfs they tended to run at about the max
> throughput of the drives as long as there wasn't any other access
> going on.  It can also do read-only scrubs to check data integrity
> sequentially across the disk, which suggests the checksums are stored
> at a lower layer and so the data can be verified without worrying
> about file fragmentation and so on.  This layering also lets btrfs
> switch "RAID modes" on the fly with half of the disk being RAID1 and
> half the disk being RAID5 and so on - each region of a disk is
> independent from the others and so mode changes only impact new
> regions until you do a full rebalance to rewrite everything.  Of
> course, zfs has its own advantages.
>

This drive is formatted with ext4.  It doesn't have LVM or anything just
straight ext4.  Given it is external, I didn't see the point of having
LVM on it and adding another layer to deal with when there is no
benefits to it.  While it does delete some files and overwrite others,
it mostly adds new files.  I'd guess it just throws them on the end but
who knows what it is really doing under the hood that neither of us
knows about.  ;-)

I did notice that is has reached 80% full.  I'm going to start figuring
out what not to backup and such pretty soon.  If I have my videos, I'm
good.  I could backup Documents and some other OS related stuff on
another drive.  I usually backup /etc and the world file.  Hmmm, may
want to grab my local overlay too.  I hadn't thought about that.  Funny
how typing in a email makes me think of things like that. 

Maybe when I do a large backup next time, I'll think to save the output
so I can share it.  That might shed some light on the situation.  I just
thought it was interesting that when it hit about 50 or 60GBs of data it
got a good deal slower and then did the bumpy thing a lot longer too. 
To me, I figured it ran out of PMR space and was slinging stuff good
trying to catch up.  There was several files where it just sat there,
for many seconds with nothing moving.  Usually, 10 seconds is about the
longest wait but it is a large file, movie or something that is GBs or
so.  I'd guess close to a minute for some this last time.  Big
difference.  Most files were around 300MBs too.  Found a source for good
HD stuff that isn't to large.  ;-)

Anyway, thought it worth mentioning.  The drive serves its purpose well
enough for what it does.  Still wouldn't want it somewhere more critical
tho.  Sticking with PMR/CMR until they get things sorted out better.  If
I bought another drive just for external backup use tho, I might get a
SMR if the price was right.  It wouldn't be my first choice tho.

Dale

:-)  :-)