Gentoo Archives: gentoo-performance

From: Alex Schuster <wonko@×××××××××.org>
To: gentoo-performance@l.g.o
Subject: Re: [gentoo-performance] Horrible performance
Date: Wed, 25 Aug 2010 20:03:19
Message-Id: 201008252138.12412.wonko@wonkology.org
In Reply to: Re: [gentoo-performance] Horrible performance by "J. Roeleveld"
J. Roeleveld writes:

> On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:
> > I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board > > Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM > > volumes, all encrypted, except for /usr/src and portage stuff. The > > system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current > > kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4 > > with desktop effects enabled, X itself takes about 30-40% of CPU > > time according to top. After system startup and login into KDE, 3.5G > > of RAM are occupied. This increases after a while, and I need swap > > space. Nothing to worry about I think. > > Encrypted filesystems can cause additional with activity, but I would > expect that to remain the same over a long period.
And I just moved my PORTAGE_TMPDIR to an unencrypted partition. Can LVM create noticeable overhead? I also resized my logical volumes a couple of times, could this lead to some LVM fragmentation?
> However, how is the write and read performance on those disks?
Here's the output of hdparm -t for all drives, 4 times. /dev/sda: (SATA system drive) Timing buffered disk reads: 118 MB in 3.08 seconds = 38.37 MB/sec Timing buffered disk reads: 194 MB in 3.11 seconds = 62.47 MB/sec Timing buffered disk reads: 322 MB in 3.01 seconds = 106.82 MB/sec Timing buffered disk reads: 244 MB in 3.00 seconds = 81.21 MB/sec /dev/sdb: (PATA master) Timing buffered disk reads: 114 MB in 3.02 seconds = 37.70 MB/sec Timing buffered disk reads: 114 MB in 3.00 seconds = 37.97 MB/sec Timing buffered disk reads: 116 MB in 3.05 seconds = 38.06 MB/sec Timing buffered disk reads: 116 MB in 3.05 seconds = 38.07 MB/sec /dev/sdc: (PATA slave) Timing buffered disk reads: 164 MB in 3.03 seconds = 54.21 MB/sec Timing buffered disk reads: 166 MB in 3.02 seconds = 55.04 MB/sec Timing buffered disk reads: 166 MB in 3.01 seconds = 55.10 MB/sec Timing buffered disk reads: 158 MB in 3.01 seconds = 52.41 MB/sec /dev/sdd: (SATA backup drive) Timing buffered disk reads: 314 MB in 3.00 seconds = 104.55 MB/sec Timing buffered disk reads: 312 MB in 3.01 seconds = 103.67 MB/sec Timing buffered disk reads: 308 MB in 3.01 seconds = 102.34 MB/sec Timing buffered disk reads: 314 MB in 3.00 seconds = 104.60 MB/sec The system drive throughput varies a lot, depending on other I/O. I
> You're running KDE4, guess you went for the default and use mysql for > "app- office/akonadi-server". > > I switched to using "sqlite" for this due to issues getting it to work > with mysql. I think this might help there?
So I only have to set the sqlite use flag and remove the mysql use flag for akonadi-server? I'm doing this now. And this also gives an example for what is going on here: I have just logged into KDE. I did not log out since yesterday, and when started a VM with vmplayer, the system swapped like crazy, I could not use it for minutes. After this, the panel did not react any more, and the desktop background did not redraw, so I logged out and in again. The VM started fine now. Well, could be faster, but maybe it was okay. Then I started answering your mail, and tried to reemerge akonadi-server, but I had a type, so portage took a long search for akomadi-server. meanwhile the dektop became quite unresponsive, load went high, and I made a screenshot [*]. If you look at the top right, gkrellm shows this above 'Proc'. The first increase at the left was after I started emerge, the 2nd at the right was after I pressed the PrtSc key.
> > Performance does not feel too bad at first. But after a while, I > > cannot even play videos during emerges. The playback stutters, > > sometimes I have pauses for several seconds. As long as there is no > > swap space occpied, it's not so bad I think. Maybe I have a probelm > > with disk I/O, and things get much worse when swapping occurs. When > > I look at iotop, I see various programs like chromium and various > > KDE applications appear. I guess that's normal, but should not be > > noticeable. Hey, there were times when I created a 2G tmpfs for > > /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered my > > swappiness to 10. This helps a little I think, because the swapping > > occurs later, the system is more responsive. > > Do you also encrypt swap?
Yes.
> Disk I/O is, in my experience, a very common cause for "freeze-ups". > Can you test with unencrypted disks to see if the issue occurs then as > well?
Yes, I can do this. It's some work, but I tried so much, why not this. I have some free space, and already have written a backup script that automatically creates LVM snapshots, decrypts them, and backs it up, so I can do this from the running system.
> > And I have similar problems when copying data between some old PATA > > drives. When I copy stuff and do a mkfs on another partition, mplayer > > sometimes stutters and hangs for ten seconds. No joy. Working with > > KDE sucks, switching dektops sometimes takes ages, and even now I am > > typing faster than kmail can display the characters. That's with am > > emerge of chromium running, with PORTAGE_NICENESS=10 and using > > ionice -c 3. Load is around 8, but sometimes gets even higher. And > > then, load suddenly drops back to lower values, as if somthing was > > blocking. Some applications swapping, maybe. > > Very possibly, maybe an idea to check which applications are hogging > the memory. If it is the swapping of the system, then this will be > caused by the most memory-hungry processes. > > Can you post the result of: "ps axu"? > This will give an indication which processes are running and using a > lot of memory.
First, here is free -m: total used free shared buffers cached Mem: 3452 3225 226 0 54 325 -/+ buffers/cache: 2844 607 Swap: 4094 935 3159 And here the output of top, sorted by memory. I think it is a similar output to ps axu, but more condensed and better readable via mail. The full ps aux output, sorted by memory, is in [2]. top - 21:23:03 up 1 day, 7 min, 11 users, load average: 0.04, 0.05, 0.02 Tasks: 418 total, 1 running, 417 sleeping, 0 stopped, 0 zombie Cpu(s): 17.8%us, 12.7%sy, 9.9%ni, 45.9%id, 13.5%wa, 0.1%hi, 0.1%si, 0.0%st Mem: 3534936k total, 3312312k used, 222624k free, 56264k buffers Swap: 4193272k total, 957908k used, 3235364k free, 334048k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25993 root 20 0 847m 475m 47m S 21 13.8 49:01.11 X 26553 wonko 20 0 1312m 235m 10m S 0 6.8 0:28.32 java 26925 wonko 20 0 698m 112m 18m S 0 3.2 3:30.75 kontact 26961 wonko 20 0 526m 79m 13m S 0 2.3 2:16.01 chrome 26566 wonko 20 0 1128m 72m 10m S 0 2.1 1:02.46 amarok 27056 wonko 20 0 871m 57m 19m S 0 1.7 0:05.07 chrome 30324 root 30 10 195m 55m 1136 S 0 1.6 0:09.46 emerge 27051 wonko 20 0 874m 50m 8668 S 0 1.5 0:14.16 chrome 26126 wonko 20 0 1028m 44m 11m S 2 1.3 9:18.94 plasma-desktop 27046 wonko 20 0 878m 41m 7324 S 0 1.2 0:16.59 chrome 27122 wonko 20 0 871m 36m 7660 S 0 1.0 0:40.03 chrome 27036 wonko 20 0 865m 29m 7492 S 0 0.8 0:00.94 chrome 27198 wonko 20 0 860m 28m 6240 S 0 0.8 0:04.76 chrome 26101 wonko 20 0 427m 28m 8672 S 4 0.8 5:27.48 kwin 26298 wonko 20 0 396m 26m 9.9m S 0 0.8 0:02.91 knotes 27766 wonko 20 0 857m 26m 5924 S 0 0.8 0:03.03 chrome 26903 root 20 0 64316 25m 288 S 0 0.7 0:00.33 screen 27203 wonko 20 0 865m 24m 5960 S 0 0.7 0:04.06 chrome 30226 wonko 20 0 367m 23m 9232 S 0 0.7 0:03.21 gwenview 26221 wonko 20 0 609m 23m 3752 S 0 0.7 0:02.55 knotify4 X takes an awful lot, then comes java, which is running only for my tvbrowser. And many many chrome processes.
> > Now I am out of ideas. I really hope someone here has one. I cannot > > work with this system any more when emerges are going on. > > Had similar issues with a desktop machine myself, managed to kill some > "features" that I wasn't using and this solved most of the problems.
I hope I can say this soon, too.
> Lets see where checking for IO-speeds and memory-usage of your apps > take us :)
Thanks for your help! I appreciate this very much. Wonko [1] http://www.wonkology.org/comp/desktop/2010-08-25_emerge-akomadi.png [2] http://www.wonkology.org/gentoo/

Replies

Subject Author
Re: [gentoo-performance] Horrible performance Joost Roeleveld <joost@××××××××.org>