1 |
What are you measuring the speed with - hdparm or rsync or ? |
2 |
|
3 |
hdparm is best for profiling just the harddisk (tallks to the interface |
4 |
and can bypass the cache depending on settings, rsync/cp/?? usually have |
5 |
the whole OS storage chain including encryption affecting throughput. |
6 |
Encryption itself can be highly variable depending on what you use and |
7 |
usually though not always includes compression before encryption. There |
8 |
are tools you can use to isolate where the slowdown occurs. atop is |
9 |
another one that may help. |
10 |
|
11 |
[test using a USB3 shingled drive on a 32 it arm system] |
12 |
|
13 |
xu4 ~ # hdparm -Tt /dev/sda |
14 |
/dev/sda: |
15 |
Timing cached reads: 1596 MB in 2.00 seconds = 798.93 MB/sec |
16 |
Timing buffered disk reads: 526 MB in 3.01 seconds = 174.99 MB/sec |
17 |
xu4 ~ # |
18 |
|
19 |
BillK |
20 |
|
21 |
On 21/8/22 06:45, Dale wrote: |
22 |
> Grant Taylor wrote: |
23 |
>> Sorry for the duplicate post. I had an email client error that |
24 |
>> accidentally caused me to hit send on the window I was composing in. |
25 |
> I figured it was something like that. ;-) |
26 |
> |
27 |
>> On 8/20/22 1:15 PM, Dale wrote: |
28 |
>>> Howdy, |
29 |
>> Hi, |
30 |
>> |
31 |
>>> Related question. Does encryption slow the read/write speeds of a |
32 |
>>> drive down a fair amount? |
33 |
>> My experience has been the opposite. I know that it's unintuitive |
34 |
>> that encryption would make things faster. But my understanding is |
35 |
>> that it alters how data is read from / written to the disk such that |
36 |
>> it's done in more optimized batches and / or optimized caching. |
37 |
>> |
38 |
>> This was so surprising that I decrypted a drive / re-encrypted a drive |
39 |
>> multiple times to compare things to come to the conclusion that |
40 |
>> encryption was noticeably better. |
41 |
>> |
42 |
>> Plus, encryption has the advantage of destroying the key rendering the |
43 |
>> drive safe to use independent of the data that was on it. |
44 |
>> |
45 |
>> N.B. The actual encryption key is encrypted with the passphrase. The |
46 |
>> passphrase isn't the encryption key itself. |
47 |
>> |
48 |
>>> This new 10TB drive is maxing out at about 49.51MB/s or so. |
49 |
>> I wonder if you are possibly running into performance issues related |
50 |
>> to shingled drives. Their raw capacity comes at a performance penalty. |
51 |
> This drive is not supposed to be SMR. It's a 10TB and according to a |
52 |
> site I looked on, none of them are SMR, yet. I found another site that |
53 |
> said it was CMR. So, pretty sure it isn't SMR. Nothing is 100% tho. I |
54 |
> might add, it's been at about that speed since I started the backup. If |
55 |
> you have a better source of info, it's a WD model WD101EDBZ-11B1DA0 drive. |
56 |
> |
57 |
> |
58 |
>>> I actually copied that from the progress of rsync and a nice sized |
59 |
>>> file. It's been running over 24 hours now so I'd think buffer and |
60 |
>>> cache would be well done with. LOL |
61 |
>> Ya, you have /probably/ exceeded the write back cache in the system's |
62 |
>> memory. |
63 |
>> |
64 |
>>> It did pass both a short and long self test. I used cryptsetup -s 512 |
65 |
>>> to encrypt with, nice password too. My rig has a FX-8350 8 core running |
66 |
>>> at 4GHz CPU and 32GBs of memory. The CPU is fairly busy. A little more |
67 |
>>> than normal anyway. Keep in mind, I have two encrypted drives connected |
68 |
>>> right now. |
69 |
>> The last time I looked at cryptsetup / LUKS, I found that there was a |
70 |
>> [kernel] process per encrypted block device. |
71 |
>> |
72 |
>> A hack that I did while testing things was to slice up a drive into |
73 |
>> multiple partitions, encrypt each one, and then re-aggregate the LUKS |
74 |
>> devices as PVs in LVM. This surprisingly was a worthwhile performance |
75 |
>> boost. |
76 |
> I noticed there is a kcrypt something thread running, a few actually but |
77 |
> it's hard to keep up since I see it on gkrellm's top process list. The |
78 |
> CPU is running at about 40% or so average but I do have mplayer, a |
79 |
> couple Firefox profiles, Seamonkey and other stuff running as well. I |
80 |
> still got plenty of CPU pedal left if needed. Having Ktorrent and |
81 |
> qbittorrent running together isn't helping. Thinking of switching |
82 |
> torrent software. Qbit does seem to use more memory tho. |
83 |
> |
84 |
> |
85 |
>>> Just curious if that speed is normal or not. |
86 |
>> I suspect that your drive is FAR more the bottleneck than the |
87 |
>> encryption itself is. There is a chance that the encryption's access |
88 |
>> pattern is exascerbating a drive performance issue. |
89 |
>> |
90 |
>>> Thoughts? |
91 |
>> Conceptually working in 512 B blocks on a drive that is natively 4 kB |
92 |
>> sectors. Thus causing the drive to do lots of extra work to account |
93 |
>> for the other seven 512 B blocks in a 4 kB sector. |
94 |
> I think the 512 has something to do with key size or something. Am I |
95 |
> wrong on that? If I need to use 256 or something, I can. My |
96 |
> understanding was that 512 was stronger than 256 as far as the |
97 |
> encryption goes. |
98 |
> |
99 |
> |
100 |
>>> P. S. The pulled drive I bought had like 60 hours on it. Dang near |
101 |
>>> new. |
102 |
>> :-) |
103 |
> I'm going to try some tests Rich mentioned after it is done doing its |
104 |
> backup. I don't want to stop it if I can avoid it. It's about half way |
105 |
> through, give or take a little. |
106 |
> |
107 |
> Dale |
108 |
> |
109 |
> :-) :-) |
110 |
> |