Re: [gentoo-performance] Horrible performance

[Joost Roeleveld]

On Wednesday 25 August 2010 21:38:10 Alex Schuster wrote:
> J. Roeleveld writes:
> > On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:
> > > I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board
> > > Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM
> > > volumes, all encrypted, except for /usr/src and portage stuff. The
> > > system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current
> > > kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4
> > > with desktop effects enabled, X itself takes about 30-40% of CPU
> > > time according to top. After system startup and login into KDE, 3.5G
> > > of RAM are occupied. This increases after a while, and I need swap
> > > space. Nothing to worry about I think.
> >
> > Encrypted filesystems can cause additional with activity, but I would
> > expect that to remain the same over a long period.
>
> And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
> Can LVM create noticeable overhead? I also resized my logical volumes a
> couple of times, could this lead to some LVM fragmentation?

Theoretically, LVM will create an additional overhead.
But I am extensibly using LVM on all my machines and haven't noticed any 
significant performance drops.

LVM-fragmentation is a definite possibility.
To defragment it, have a look at the following:

http://bisqwit.iki.fi/source/lvm2defrag.html
http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-defrag-
or-move-based-on-logical-volumes-689335/

I played with the first one on an older machine once and it does work quite 
nicely.

> > However, how is the write and read performance on those disks?
>
> Here's the output of hdparm -t for all drives, 4 times.
>
> /dev/sda: (SATA system drive)
>  Timing buffered disk reads:  118 MB in  3.08 seconds =  38.37 MB/sec
>  Timing buffered disk reads:  194 MB in  3.11 seconds =  62.47 MB/sec
>  Timing buffered disk reads:  322 MB in  3.01 seconds = 106.82 MB/sec
>  Timing buffered disk reads:  244 MB in  3.00 seconds =  81.21 MB/sec
>
> /dev/sdb: (PATA master)
>  Timing buffered disk reads:  114 MB in  3.02 seconds =  37.70 MB/sec
>  Timing buffered disk reads:  114 MB in  3.00 seconds =  37.97 MB/sec
>  Timing buffered disk reads:  116 MB in  3.05 seconds =  38.06 MB/sec
>  Timing buffered disk reads:  116 MB in  3.05 seconds =  38.07 MB/sec
>
> /dev/sdc: (PATA slave)
>  Timing buffered disk reads:  164 MB in  3.03 seconds =  54.21 MB/sec
>  Timing buffered disk reads:  166 MB in  3.02 seconds =  55.04 MB/sec
>  Timing buffered disk reads:  166 MB in  3.01 seconds =  55.10 MB/sec
>  Timing buffered disk reads:  158 MB in  3.01 seconds =  52.41 MB/sec
>
> /dev/sdd: (SATA backup drive)
>  Timing buffered disk reads:  314 MB in  3.00 seconds = 104.55 MB/sec
>  Timing buffered disk reads:  312 MB in  3.01 seconds = 103.67 MB/sec
>  Timing buffered disk reads:  308 MB in  3.01 seconds = 102.34 MB/sec
>  Timing buffered disk reads:  314 MB in  3.00 seconds = 104.60 MB/sec
>
> The system drive throughput varies a lot, depending on other I/O.

Those look ok to me, except that I would expect SATA-drives to be faster then 
PATA drives.

> > You're running KDE4, guess you went for the default and use mysql for
> > "app- office/akonadi-server".
> >
> > I switched to using "sqlite" for this due to issues getting it to work
> > with mysql. I think this might help there?
>
> So I only have to set the sqlite use flag and remove the mysql use flag
> for akonadi-server? I'm doing this now. And this also gives an example for
> what is going on here:

And unset mysql.
There is one issue that needs to be resolved manually with getting it to work 
with sqlite.
See: 
http://forums.gentoo.org/viewtopic-t-834883-view-
next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4

Here is the bit that will solve the problem:
--
$ cd ~/.local/share/akonadi
$ sqlite3 akonadi.db
sqlite> INSERT INTO ResourceTable (name, isVirtual) VALUES 
('akonadi_search_resource', 1);
sqlite> .exit 
--

> I have just logged into KDE. I did not log out since yesterday, and when
> started a VM with vmplayer, the system swapped like crazy, I could not use
> it for minutes. After this, the panel did not react any more, and the
> desktop background did not redraw, so I logged out and in again. The VM
> started fine now. Well, could be faster, but maybe it was okay.
> Then I started answering your mail, and tried to reemerge akonadi-server,
> but I had a type, so portage took a long search for akomadi-server.
> meanwhile the dektop became quite unresponsive, load went high, and I made
> a screenshot [*]. If you look at the top right, gkrellm shows this above
> 'Proc'. The first increase at the left was after I started emerge, the 2nd
> at the right was after I pressed the PrtSc key.

VMWare allows virtual machines to use more memory then is actually available.
Also, there are settings in VMWare (possibly enabled by default) that cause 
the memory to be duplicated onto disk.
This can cause issues like you are seeing.

> > > Performance does not feel too bad at first. But after a while, I
> > > cannot even play videos during emerges. The playback stutters,
> > > sometimes I have pauses for several seconds. As long as there is no
> > > swap space occpied, it's not so bad I think. Maybe I have a probelm
> > > with disk I/O, and things get much worse when swapping occurs. When
> > > I look at iotop, I see various programs like chromium and various
> > > KDE applications appear. I guess that's normal, but should not be
> > > noticeable. Hey, there were times when I created a 2G tmpfs for
> > > /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered my
> > > swappiness to 10. This helps a little I think, because the swapping
> > > occurs later, the system is more responsive.
> >
> > Do you also encrypt swap?
>
> Yes.
>
> > Disk I/O is, in my experience, a very common cause for "freeze-ups".
> > Can you test with unencrypted disks to see if the issue occurs then as
> > well?
>
> Yes, I can do this. It's some work, but I tried so much, why not this. I
> have some free space, and already have written a backup script that
> automatically creates LVM snapshots, decrypts them, and backs it up, so I
> can do this from the running system.

Ok, am interested to see if running unencrypted actually has benefits here.

> > > And I have similar problems when copying data between some old PATA
> > > drives. When I copy stuff and do a mkfs on another partition, mplayer
> > > sometimes stutters and hangs for ten seconds. No joy. Working with
> > > KDE sucks, switching dektops sometimes takes ages, and even now I am
> > > typing faster than kmail can display the characters. That's with am
> > > emerge of chromium running, with PORTAGE_NICENESS=10 and using
> > > ionice -c 3. Load is around 8, but sometimes gets even higher. And
> > > then, load suddenly drops back to lower values, as if somthing was
> > > blocking. Some applications swapping, maybe.
> >
> > Very possibly, maybe an idea to check which applications are hogging
> > the memory. If it is the swapping of the system, then this will be
> > caused by the most memory-hungry processes.
> >
> > Can you post the result of: "ps axu"?
> > This will give an indication which processes are running and using a
> > lot of memory.
>
> First, here is free -m:
>              total       used       free     shared    buffers     cached
> Mem:          3452       3225        226          0         54        325
> -/+ buffers/cache:       2844        607
> Swap:         4094        935       3159
>
> And here the output of top, sorted by memory. I think it is a similar
> output to ps axu, but more condensed and better readable via mail. The
> full ps aux output, sorted by memory, is in [2].
>
<snipped top>
>
> X takes an awful lot, then comes java, which is running only for my
> tvbrowser. And many many chrome processes.

How many web browser windows do you have open? :)

Also, do you have file indexing enabled? 

> > > Now I am out of ideas. I really hope someone here has one. I cannot
> > > work with this system any more when emerges are going on.
> >
> > Had similar issues with a desktop machine myself, managed to kill some
> > "features" that I wasn't using and this solved most of the problems.
>
> I hope I can say this soon, too.

In my experience, X uses more memory when a lot of windows are open.
And yours uses about 4 times as much as mine.
But then again, I don't have much running at the moment.

> > Lets see where checking for IO-speeds and memory-usage of your apps
> > take us :)
>
> Thanks for your help! I appreciate this very much.

I'll do my best :)

--
Joost