Re: [gentoo-user] Re: Portage performance dropped considerably

[Andrew Savchenko]

On Fri, 31 Jan 2014 19:13:21 +0000 Mick wrote:
> On Friday 31 Jan 2014 19:03:05 Andrew Savchenko wrote:
> > On Sun, 26 Jan 2014 20:30:19 +0200 Alan McKinnon wrote:
[...]
> > > I'm willing to give up 4 minutes while emerge runs so I don't have to
> > > spend many more minutes right afterwards doing manually the very shit
> > > that software is very good at. Whether portage is a complete pile of
> > > dogshit software or not is beside the point. Even if it is, my 4 minutes
> > > still buys me lots <shrug>
> >
> > 4 minutes are expendable but... on Atom N270 (my laptop) emerge
> > -DNuav world takes 40 (yes, forty) minutes to build dependency tree
> > with sqlite cache enabled and 60 minutes without sqlite. System was
> > pretty old (not updated aside from GLSA updates for a year). And this
> > 40 minutes repeated many times since USE flag clashes and dependency
> > resolution failures. So I spent may day, damn whole day(!) for the
> > sake to just start compiling (distcc is my friend here).
>
> Out of interest what fs are you running portage on?  I changed an old box from
> reiserfs to ext4 and couldn't believe the speed up I got.

I use ext4. In my case the problem is in CPU usage and Atom N270 is
pretty weak these days. On this box HDD is fast and NCQ capable,
memory is enough (2GB for 32bit setup). During dependencies
calculation all 100% of time is spent in the python process. CPU is
slightly overclocked (as much as SHE allows). There is nothing more
that I can do here as an admin of this host. (Gentoo setup itself is
complicated with >2200 packages.)

IMO the only way to improve this issue (without throwing good working
hardware in the window) is to rewrite dependency resolution code in
some highly optimized pure C library (probably with some asm code)
and to use this library via some python binding from portage. But I
suppose algorithm itself should be reviewed first.

Best regards,
Andrew Savchenko