Gentoo Archives: gentoo-user

From: Dale <rdalek1967@×××××.com>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] External hard drive and idle activity
Date: Fri, 03 Jan 2020 00:37:43
Message-Id: 3a9d0fde-6cd7-b0cf-72fa-7ef1625ddb5e@gmail.com
In Reply to: Re: [gentoo-user] External hard drive and idle activity by Rich Freeman
1 Rich Freeman wrote:
2 > On Thu, Jan 2, 2020 at 1:41 PM Dale <rdalek1967@×××××.com> wrote:
3 >> Rich Freeman wrote:
4 >>> Out of curiosity, what model drive is it? Is it by chance an SMR /
5 >>> archive drive?
6 >> Device Model: ST8000AS0003-2HH188
7 >>
8 >> I recall reading about SMR but can't recall the details of what it is.
9 >> As far as I know, this is just a basic 8TB drive.
10 > This is an SMR drive. You should DEFINITELY read up on what they are.
11 >
12 > For reads they're completely normal. For sequential writes to unused
13 > space they're completely normal. For random writes or overwrites they
14 > are significantly different from traditional hard drives.
15 >
16 > They work a bit like an SSD in the sense that blocks are arranged into
17 > larger erase regions. Within a region blocks can only be written
18 > sequentially. If you want to overwrite one block in the middle of a
19 > region, the drive will read the entire region into RAM, then write the
20 > entire region sequentially with the overwritten block to a new spot on
21 > the disk. This is just like in an SSD where if try to overwrite a
22 > block in a region with any unTRIMmed blocks the drive must read the
23 > entire region, erase the region, and write the modified region.
24 >
25 > Except that in an SSD those extra reads/writes operate with SSD access
26 > times. With an SMR drive those extra reads/writes operate with hard
27 > drive latencies, so they're MUCH more costly.
28 >
29 > For backup use they're usually fine, IF you're writing in a sequential
30 > file format that is appended to. If you're using rsync to do your
31 > backups then that isn't what you're doing and you're probably paying a
32 > heavy penalty. If you were doing incremental backups using
33 > tar/duplicity/whatever then you'd probably be fine.
34 >
35 > Some filesystems might be optimized for these drives to reduce the
36 > amount of overwriting in place. I haven't looked into it. I'd expect
37 > a log-based filesystem to work fairly well, though those can have high
38 > levels of fragmentation which is better suited for SSD than SMR.
39 >
40 > These drives all have fairly active firmware that manages this special
41 > overwrite process so that they can be used with operating systems that
42 > are naive to how they work. I wouldn't be surprised if this is what
43 > is causing the drive to be active after you unmount it. In theory it
44 > should be harmless to power it off. However, leaving it powered on
45 > probably will improve its performance as it can take care of any
46 > garbage collection before the next time you use it. If whatever
47 > journal it is using to speed things up gets full then you'll feel the
48 > full brunt of any write penalties until it is flushed.
49 >
50 > You might want to seriously consider changing to a backup format that
51 > just creates big tail-appended files containing incremental changes.
52 > Something like rsync that just outputs bazillions of little files is
53 > going to create lots of random writes when things change, vs
54 > consolidating all those changes into one file that just grows at the
55 > end. Treat them the way you would a tape (which is what tar was
56 > designed for).
57 >
58 > Nothing wrong with SMR drives per se - they can potentially be cheaper
59 > especially for backup (using an appropriate file format), and are just
60 > as fast for reading so they're also great for infrequently changing
61 > bulky data. However, random writes are very costly and you should be
62 > aware of that going in...
63 >
64
65
66 I had no idea this was that type of drive.  I wonder if any of my other
67 drives are this type as well.  If it matters, I use ext4 on my drives,
68 except /boot, but I've thought about using something that is also
69 compatible with windoze, just in case I need to copy some things to
70 someone else's system.  Thing is, I'm not real big on those file systems
71 since I lose some info such as permissions and all.  Does my using ext4
72 change this any?  Better or worse?
73
74 Should I change the mounting options for this drive?  I've read some
75 people use certain options for SSD and they are needed.  This is the
76 options being used according to mount:
77
78 /dev/sdj1 on /run/media/dale/8tb-backup type ext4
79 (rw,nosuid,nodev,relatime,uhelper=udisks2)
80
81 That I guess is the default way KDE/udisks/etc does it. 
82
83 One other question, what should I look for to avoid these types of
84 drives?  I'll go back and look but I don't recall it saying anything
85 about this.  I'm also trying to figure out if this is a good thing or not. 
86
87 Thanks much for the info.  I'll read this one again shortly. Lot of info
88 to absorb. 
89
90 Dale
91
92 :-)  :-) 

Replies

Subject Author
Re: [gentoo-user] External hard drive and idle activity Dale <rdalek1967@×××××.com>