1 |
On Fri, Nov 26, 2010 at 7:57 AM, Stroller |
2 |
<stroller@××××××××××××××××××.uk> wrote: |
3 |
> Hi there, |
4 |
> |
5 |
> As per subject, what's the best way to improve interactivity with heavy disk activity, please? |
6 |
> |
7 |
> Or perhaps a better question would be: what approaches are available? |
8 |
> |
9 |
> Presently my main Linux system is basically just a storage server with a *really slow* disk controller. I do all my web-browsing and email (and most other things) on my Mac laptop (because my Mac desktop has recently died ☹), but I occasionally do some bash or perl scripting, searches and other stuff on this Linux box. |
10 |
> |
11 |
> Normally this isn't a problem - the machine is an old Pentium 4 but plenty powerful enough for this simple command-line stuff. However I have recently bought a new STB which plays DVD .iso files across the network, so I started ripping DVDs on storage server, using dvdbackup && mkisofs. When I do so, interactivity becomes *dire* - it takes maybe 15 seconds for *any* command to execute. |
12 |
> |
13 |
> My immediate reaction was to consider the recent "200-line patch to kernel => superkernel" thread: |
14 |
> http://www.gossamer-threads.com/lists/gentoo/user/221770 |
15 |
> |
16 |
> But I have also heard of `ionice` in the past: http://linux.die.net/man/1/ionice |
17 |
> |
18 |
> I've never used that - in fact, I can't recall ever having to use the regular `nice` - but I think maybe I should consider it. |
19 |
> |
20 |
> Does anyone have any thoughts, please? |
21 |
> |
22 |
> Stroller. |
23 |
|
24 |
Hi Stroller, |
25 |
Let me respond in a different direction with a different focus. |
26 |
This response is could and should be used in parallel with the other |
27 |
responses you've received already. |
28 |
|
29 |
One problem I ran into in the last year was very similar to to |
30 |
symptoms you are relating, although my machine was quite different in |
31 |
terms of configuration. It's a 6 core/12 thread machine with 24GB of |
32 |
DRAM and both RAID1 and RAID0. It's not slow but at times, basically |
33 |
when copying some large files, the machine ground to a halt for along |
34 |
periods of time - sometimes over 1 or 2 minutes! I didn't know why but |
35 |
eventually did find some good info that helped a lot. |
36 |
|
37 |
Here's the basic issue with the system. I am copying a large file, |
38 |
so it's being read, portions are being held in memory for a while, and |
39 |
then written out to disk as the kernel decides it's appropriate. The |
40 |
question is how does this work? And why, once it starts, does it take |
41 |
so long to complete and what's up with the bad interactivity with the |
42 |
machine. |
43 |
|
44 |
The answer goes essentially like this: |
45 |
|
46 |
1) The kernel starts the read (in your case possibly the rip) and it |
47 |
begins filling up memory. It does NOT start writing the data out. |
48 |
|
49 |
2) Two parameters (vm.dirty_background_ratio & vm.dirty_ratio) tell |
50 |
the kernel how to wait to start the write. |
51 |
|
52 |
vm.dirty_background_ratio is how much memory will be used to store the |
53 |
copy before the write begins |
54 |
|
55 |
vm.dirty_ratio is how low the data in memory must go below before the |
56 |
kernel lowers the priority of this task and lets other things (like |
57 |
desktop stuff) get access to disks, etc. |
58 |
|
59 |
Check your current settings: |
60 |
|
61 |
c2stable ~ # sysctl -a | grep vm.dirty |
62 |
error: "Invalid argument" reading key "fs.binfmt_misc.register" |
63 |
error: permission denied on key 'net.ipv4.route.flush' |
64 |
vm.dirty_background_ratio = 5 |
65 |
vm.dirty_background_bytes = 0 |
66 |
vm.dirty_ratio = 10 |
67 |
vm.dirty_bytes = 0 |
68 |
vm.dirty_writeback_centisecs = 500 |
69 |
vm.dirty_expire_centisecs = 3000 |
70 |
error: permission denied on key 'net.ipv6.route.flush' |
71 |
c2stable ~ # |
72 |
|
73 |
Now, if you have, for instance, 8GB or memory, and |
74 |
vm.dirty_background_ratio is set to (for instance) 10%, then that |
75 |
means you are saving 80MB of the copied file in memory before it |
76 |
starts writing to disk and you've got to get below 40MB (5%) before |
77 |
the kernel even starts to lower the priority and lets the system |
78 |
operate more normally. |
79 |
|
80 |
Now, if you have a slow disk system it takes longer, once the write |
81 |
finally starts, for the kernel to get down to the low level and start |
82 |
to let the system operate normally again. One the other hand, even if |
83 |
you have fast disks like my system, but you have TONS of memory, then |
84 |
10% of 12GB is 1.2GB and it takes a LONG time to write 1.2GB to disk |
85 |
even if the disks are fast. |
86 |
|
87 |
There are threads out there that I was involved in, but there's |
88 |
probably a better place to read is a site I liked when I was dealing |
89 |
with this: |
90 |
|
91 |
http://hep.kbfi.ee/index.php/IT/KernelTuning |
92 |
|
93 |
He shows a lot of commands and goes through some decisions you might |
94 |
want to think about. |
95 |
|
96 |
Even if it's not your answer it was, for me at the time, and |
97 |
interesting learning experience. |
98 |
|
99 |
Hope this helps, |
100 |
Mark |