Re: [gentoo-performance] Horrible performance

[Alex Schuster]

Joost Roeleveld writes:

> On Wednesday 25 August 2010 21:38:10 Alex Schuster wrote:

> > And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
> > Can LVM create noticeable overhead? I also resized my logical volumes
> > a couple of times, could this lead to some LVM fragmentation?
>
> Theoretically, LVM will create an additional overhead.
> But I am extensibly using LVM on all my machines and haven't noticed
> any significant performance drops.

That's also what I heard.

> LVM-fragmentation is a definite possibility.
> To defragment it, have a look at the following:
>
> http://bisqwit.iki.fi/source/lvm2defrag.html

Cool! 

> http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-
> defrag- or-move-based-on-logical-volumes-689335/
>
> I played with the first one on an older machine once and it does work
> quite nicely.

Any idea how to check how bad the fragmentation actually is?

BTW:
wonko@weird ~ $ mount | wc -l
48

I use LVM for about everything now, it makes things so much easier. First, 
I had two volume groups on my system drive, one for the system, placed at 
the front where the drive is supposed to be faster, and one for data. But 
I don't do this any more, it cuts down flexibility, and is probably not 
worth the effort.

> > > However, how is the write and read performance on those disks?
> >
> > Here's the output of hdparm -t for all drives, 4 times.
> >
> > /dev/sda: (SATA system drive)
> >
> >  Timing buffered disk reads:  118 MB in  3.08 seconds =  38.37 MB/sec
> >  Timing buffered disk reads:  194 MB in  3.11 seconds =  62.47 MB/sec
> >  Timing buffered disk reads:  322 MB in  3.01 seconds = 106.82 MB/sec
> >  Timing buffered disk reads:  244 MB in  3.00 seconds =  81.21 MB/sec
> >
> > /dev/sdb: (PATA master)
> >
> >  Timing buffered disk reads:  114 MB in  3.02 seconds =  37.70 MB/sec
> >  Timing buffered disk reads:  114 MB in  3.00 seconds =  37.97 MB/sec
> >  Timing buffered disk reads:  116 MB in  3.05 seconds =  38.06 MB/sec
> >  Timing buffered disk reads:  116 MB in  3.05 seconds =  38.07 MB/sec
> >
> > /dev/sdc: (PATA slave)
> >
> >  Timing buffered disk reads:  164 MB in  3.03 seconds =  54.21 MB/sec
> >  Timing buffered disk reads:  166 MB in  3.02 seconds =  55.04 MB/sec
> >  Timing buffered disk reads:  166 MB in  3.01 seconds =  55.10 MB/sec
> >  Timing buffered disk reads:  158 MB in  3.01 seconds =  52.41 MB/sec
> >
> > /dev/sdd: (SATA backup drive)
> >
> >  Timing buffered disk reads:  314 MB in  3.00 seconds = 104.55 MB/sec
> >  Timing buffered disk reads:  312 MB in  3.01 seconds = 103.67 MB/sec
> >  Timing buffered disk reads:  308 MB in  3.01 seconds = 102.34 MB/sec
> >  Timing buffered disk reads:  314 MB in  3.00 seconds = 104.60 MB/sec
> >
> > The system drive throughput varies a lot, depending on other I/O.
>
> Those look ok to me, except that I would expect SATA-drives to be
> faster then PATA drives.

Well, they are, except for the sometimes busy system drive. And by now I 
get similar results as for the 2nd SATA drive, throughput is between 90 
and 110 MB/sec.


> > > You're running KDE4, guess you went for the default and use mysql
> > > for "app- office/akonadi-server".
> > >
> > > I switched to using "sqlite" for this due to issues getting it to
> > > work with mysql. I think this might help there?
> >
> > So I only have to set the sqlite use flag and remove the mysql use
> > flag for akonadi-server? I'm doing this now. And this also gives an
> > example for what is going on here:
> And unset mysql.
> There is one issue that needs to be resolved manually with getting it
> to work with sqlite.
> See:
> http://forums.gentoo.org/viewtopic-t-834883-view-
> next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4
>
> Here is the bit that will solve the problem:
> --
> $ cd ~/.local/share/akonadi
> $ sqlite3 akonadi.db
> sqlite> INSERT INTO ResourceTable (name, isVirtual) VALUES
> ('akonadi_search_resource', 1);
> sqlite> .exit

I did nothing about this by now, but I probabyl will soon. Thanks for the 
tip!
I also had trouble with akonadi in the past, and it still gives 
warnings/errors at every startup, but at least it seems to work now.


> > > Disk I/O is, in my experience, a very common cause for
> > > "freeze-ups". Can you test with unencrypted disks to see if the
> > > issue occurs then as well?
> >
> > Yes, I can do this. It's some work, but I tried so much, why not
> > this. I have some free space, and already have written a backup
> > script that automatically creates LVM snapshots, decrypts them, and
> > backs it up, so I can do this from the running system.
>
> Ok, am interested to see if running unencrypted actually has benefits
> here.

Me too, but as things are quite faster now already, the priority for this 
task is much lower than it was yesterday :)


> > > Can you post the result of: "ps axu"?
> > > This will give an indication which processes are running and using
> > > a lot of memory.
> >
> > First, here is free -m:
> >              total       used       free     shared    buffers
> >              cached
> >
> > Mem:          3452       3225        226          0         54
> > 325 -/+ buffers/cache:       2844        607
> > Swap:         4094        935       3159
> >
> > And here the output of top, sorted by memory. I think it is a similar
> > output to ps axu, but more condensed and better readable via mail.
> > The full ps aux output, sorted by memory, is in [2].
>
> <snipped top>
>
> > X takes an awful lot, then comes java, which is running only for my
> > tvbrowser. And many many chrome processes.
>
> How many web browser windows do you have open? :)

wonko@weird ~ $ ps ax | grep chromium-browser | wc -l
42

Well, that's tabs, not windows. Chromium uses one process for each tab. 
I'm not sure if I really want to use Chromium, I started using it because 
of a nasty konqueror bug that made KDE freeze occasionally, but it seems 
to be fixed now.

> Also, do you have file indexing enabled?

Do you mean Strigi/Nepomuk? I just turned it on again today. It never 
worked, but crashed. In the last days I investigated this further and 
filed two bugs about this, and they were already fixed. I'm running the 
git version now, and so far it seems to work fine, but my home directory 
is not fully indexed yet. And there were also issues with my music folder, 
let's see if indexing this will still make strigi crash.
I should have done this before - I am waiting over a year now for a fixed 
version, but apparently few people if any have those crashes, so I had to 
report them myself.
At the moment it's indexing, but so far system load is not very high. 
That's much better than a year ago when I used it the last time.


> > > > Now I am out of ideas. I really hope someone here has one. I
> > > > cannot work with this system any more when emerges are going on.
> > >
> > > Had similar issues with a desktop machine myself, managed to kill
> > > some "features" that I wasn't using and this solved most of the
> > > problems.
> >
> > I hope I can say this soon, too.
>
> In my experience, X uses more memory when a lot of windows are open.
> And yours uses about 4 times as much as mine.
> But then again, I don't have much running at the moment.

I like to have many things open. With enough memory it's no problem. 
Unless you are using ati-drivers, that is.

	Wonko