Gentoo Archives: gentoo-user

From: Rich Freeman <rich0@g.o>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] External hard drive and idle activity
Date: Thu, 02 Jan 2020 19:12:24
Message-Id: CAGfcS_mf1QqNyKfTb_A31b9be52pHT1XnAMDMHRhFTs_Tabq4w@mail.gmail.com
In Reply to: Re: [gentoo-user] External hard drive and idle activity by Dale
1 On Thu, Jan 2, 2020 at 1:41 PM Dale <rdalek1967@×××××.com> wrote:
2 >
3 > Rich Freeman wrote:
4 > >
5 > > Out of curiosity, what model drive is it? Is it by chance an SMR /
6 > > archive drive?
7 >
8 > Device Model: ST8000AS0003-2HH188
9 >
10 > I recall reading about SMR but can't recall the details of what it is.
11 > As far as I know, this is just a basic 8TB drive.
12
13 This is an SMR drive. You should DEFINITELY read up on what they are.
14
15 For reads they're completely normal. For sequential writes to unused
16 space they're completely normal. For random writes or overwrites they
17 are significantly different from traditional hard drives.
18
19 They work a bit like an SSD in the sense that blocks are arranged into
20 larger erase regions. Within a region blocks can only be written
21 sequentially. If you want to overwrite one block in the middle of a
22 region, the drive will read the entire region into RAM, then write the
23 entire region sequentially with the overwritten block to a new spot on
24 the disk. This is just like in an SSD where if try to overwrite a
25 block in a region with any unTRIMmed blocks the drive must read the
26 entire region, erase the region, and write the modified region.
27
28 Except that in an SSD those extra reads/writes operate with SSD access
29 times. With an SMR drive those extra reads/writes operate with hard
30 drive latencies, so they're MUCH more costly.
31
32 For backup use they're usually fine, IF you're writing in a sequential
33 file format that is appended to. If you're using rsync to do your
34 backups then that isn't what you're doing and you're probably paying a
35 heavy penalty. If you were doing incremental backups using
36 tar/duplicity/whatever then you'd probably be fine.
37
38 Some filesystems might be optimized for these drives to reduce the
39 amount of overwriting in place. I haven't looked into it. I'd expect
40 a log-based filesystem to work fairly well, though those can have high
41 levels of fragmentation which is better suited for SSD than SMR.
42
43 These drives all have fairly active firmware that manages this special
44 overwrite process so that they can be used with operating systems that
45 are naive to how they work. I wouldn't be surprised if this is what
46 is causing the drive to be active after you unmount it. In theory it
47 should be harmless to power it off. However, leaving it powered on
48 probably will improve its performance as it can take care of any
49 garbage collection before the next time you use it. If whatever
50 journal it is using to speed things up gets full then you'll feel the
51 full brunt of any write penalties until it is flushed.
52
53 You might want to seriously consider changing to a backup format that
54 just creates big tail-appended files containing incremental changes.
55 Something like rsync that just outputs bazillions of little files is
56 going to create lots of random writes when things change, vs
57 consolidating all those changes into one file that just grows at the
58 end. Treat them the way you would a tape (which is what tar was
59 designed for).
60
61 Nothing wrong with SMR drives per se - they can potentially be cheaper
62 especially for backup (using an appropriate file format), and are just
63 as fast for reading so they're also great for infrequently changing
64 bulky data. However, random writes are very costly and you should be
65 aware of that going in...
66
67 --
68 Rich

Replies

Subject Author
Re: [gentoo-user] External hard drive and idle activity madscientistatlarge <madscientistatlarge@××××××××××.com>
Re: [gentoo-user] External hard drive and idle activity Dale <rdalek1967@×××××.com>
Re: [gentoo-user] External hard drive and idle activity Wol <antlists@××××××××××××.uk>
[gentoo-user] Re: External hard drive and idle activity Ian Zimmerman <itz@××××××××××××.org>