Re: [gentoo-performance] Horrible performance

[J. Roeleveld]

On Thursday 26 August 2010 17:48:44 Alex Schuster wrote:
> Joost Roeleveld writes:
> > LVM-fragmentation is a definite possibility.
> > To defragment it, have a look at the following:
> >
> > http://bisqwit.iki.fi/source/lvm2defrag.html
>
> Cool!

Do test it first and check what it wants to do.
Basically, it moves all the blocks around untill you have them all in the 
sequence you want them.
There are some limitations, but it worked when I tested it.
Btw, I provide no warranty what-so-ever, especially as I did not write any 
part of it :)

> > http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-
> > defrag- or-move-based-on-logical-volumes-689335/
> >
> > I played with the first one on an older machine once and it does work
> > quite nicely.
>
> Any idea how to check how bad the fragmentation actually is?

Yes:
desktop ~ #  lvs -o lv_name,seg_pe_ranges
  LV         PE Ranges            
  distfiles  /dev/sda4:37632-39423
  home       /dev/sda4:9472-12031 
  home       /dev/sda4:39424-40703
  home       /dev/sda4:42240-44799
  opt        /dev/sda4:8192-9471  
  opt        /dev/sda4:40704-42239
  packages   /dev/sda4:45312-47359
  portage    /dev/sda4:3072-3327  
  swap       /dev/sda4:0-3071     
  tmp        /dev/sda4:3328-4607  
  usr        /dev/sda4:6400-8191  
  usr        /dev/sda4:81664-81919
  usr        /dev/sda4:44800-45311
  var        /dev/sda4:4608-5119  
  vartmp     /dev/sda4:5120-6399  
  vartmp     /dev/sda4:60160-61183
  virtualbox /dev/sda4:12032-32511
  virtualbox /dev/sda4:76544-81663
  virtualbox /dev/sda4:58112-60159
  virtualbox /dev/sda4:81920-83310
  virtualbox /dev/sda4:47360-58111
  virtualbox /dev/sda4:32512-33680

home, vartmp and virtualbox are fragmented into different sections.

Alternatively, the first part of that script actually generates a text-file 
which you then need to edit into the next text-file.
The first one actually shows you how the different parts are laid out.

> BTW:
> wonko@weird ~ $ mount | wc -l
> 48

desktop ~ # lvs | wc -l
12

server ~ # lvs | wc -l
99
(Ok, this one runs virtual machines with Xen, but online-resizing works and 
xen-virtual machines get notified of the new size)

> I use LVM for about everything now, it makes things so much easier. First,
> I had two volume groups on my system drive, one for the system, placed at
> the front where the drive is supposed to be faster, and one for data. But
> I don't do this any more, it cuts down flexibility, and is probably not
> worth the effort.

I don't think it's worth the effort as well. You can still move the LVs around 
physically using the "lvm-defrag" tool.
It's very verbose by nature as it doesn't do any changes untill you tell it 
to. And you're the one starting the final script.

That tool basically generates a script that calls "pvmove" a couple of times. 
And the script even contains comments describing what it is doing for each 
step.
It does expect the user to determine which LVs end up where.

> > Those look ok to me, except that I would expect SATA-drives to be
> > faster then PATA drives.
>
> Well, they are, except for the sometimes busy system drive. And by now I
> get similar results as for the 2nd SATA drive, throughput is between 90
> and 110 MB/sec.

Btw, I tend to use "hdparm -Tt <device>" to do the testing:
desktop ~ # hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   3456 MB in  2.00 seconds = 1728.51 MB/sec
 Timing buffered disk reads:  276 MB in  3.01 seconds =  91.75 MB/sec

> > And unset mysql.
> > There is one issue that needs to be resolved manually with getting it
> > to work with sqlite.
> > See:
> > http://forums.gentoo.org/viewtopic-t-834883-view-
> > next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4
> >
> > Here is the bit that will solve the problem:
> > --
> > $ cd ~/.local/share/akonadi
> > $ sqlite3 akonadi.db
> > sqlite> INSERT INTO ResourceTable (name, isVirtual) VALUES
> > ('akonadi_search_resource', 1);
> > sqlite> .exit
>
> I did nothing about this by now, but I probabyl will soon. Thanks for the
> tip!
> I also had trouble with akonadi in the past, and it still gives
> warnings/errors at every startup, but at least it seems to work now.

It doesn't complain for me anymore since I switched to sqlite.
Main reason for that: I don't like MySQL and prefer not to run a full database 
on my desktop anyway. And configuring it to use the Database on the server goes 
too far.

> > > > Disk I/O is, in my experience, a very common cause for
> > > > "freeze-ups". Can you test with unencrypted disks to see if the
> > > > issue occurs then as well?
> > >
> > > Yes, I can do this. It's some work, but I tried so much, why not
> > > this. I have some free space, and already have written a backup
> > > script that automatically creates LVM snapshots, decrypts them, and
> > > backs it up, so I can do this from the running system.
> >
> > Ok, am interested to see if running unencrypted actually has benefits
> > here.
>
> Me too, but as things are quite faster now already, the priority for this
> task is much lower than it was yesterday :)

Ofcourse, depending on the encryption-method used, the performance impact can 
be a lot to not much at all.

> > > X takes an awful lot, then comes java, which is running only for my
> > > tvbrowser. And many many chrome processes.
> >
> > How many web browser windows do you have open? :)
>
> wonko@weird ~ $ ps ax | grep chromium-browser | wc -l
> 42
>
> Well, that's tabs, not windows. Chromium uses one process for each tab.
> I'm not sure if I really want to use Chromium, I started using it because
> of a nasty konqueror bug that made KDE freeze occasionally, but it seems
> to be fixed now.

I actually use Firefox myself and not had any real issues.

> > Also, do you have file indexing enabled?
>
> Do you mean Strigi/Nepomuk? I just turned it on again today. It never
> worked, but crashed. In the last days I investigated this further and
> filed two bugs about this, and they were already fixed. I'm running the
> git version now, and so far it seems to work fine, but my home directory
> is not fully indexed yet. And there were also issues with my music folder,
> let's see if indexing this will still make strigi crash.
> I should have done this before - I am waiting over a year now for a fixed
> version, but apparently few people if any have those crashes, so I had to
> report them myself.
> At the moment it's indexing, but so far system load is not very high.
> That's much better than a year ago when I used it the last time.

I haven't managed to get it to work to the point I can actually use it.
Maybe I need to read the docs to see how people expect it to work.

I actually archive my files in such a way that I can easily find them back.
Comes from not having decent indexing tools when I started with computers and 
having a lot of data to keep.

> > In my experience, X uses more memory when a lot of windows are open.
> > And yours uses about 4 times as much as mine.
> > But then again, I don't have much running at the moment.
>
> I like to have many things open. With enough memory it's no problem.
> Unless you are using ati-drivers, that is.

Hehe, if it isn't ati-drivers, it will be something else causing problems.

--
Joost