Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 13:12:56 Zac Medico wrote:
> On 01/17/2015 03:35 AM, Patrick Lauer wrote:
> > * Portage is too slow
> >
> >     On 'small' hardware emerge -upNDv @world can take enough time
> >     to make updates prohibitive - on an 800Mhz machine it took me
> >     about 3 days to figure out a solution to some silly blockers due to
> >     the
> >     very slow change test cycle
> >     Fix: Do some profiling and un-refactoring to remove useless code
> >     Fix: Set up a reference system (CI) to catch future regressions
>
> I'm certainly in favor of improving portage performance. However, for
> "small" hardware (which includes many ARM and MIPS systems), we really
> need to focus on binary package support. Note that dependency resolution
> for installing binary packages tends to be much simpler than for
> building ebuilds from source, and this translates into much shorter
> dependency resolution time.
>
That's an orthogonal problem. And that's coming from someone who extensively 
uses binpkgs already ...

The problem with (dependency resolution) performance is that in some 
scenarios, even on fast machines, it takes "too long" - especially if there's 
some silly useflag mismatch and two packages that have ~arch keywords, and now 
you need 12 attempts to figure out a solution that works for you ...
At 30 seconds for each resolution cycle that's 6 minutes waiting for portage.

If it were faster it'd almost be interactive ...

Also - just for comparison:
On my currently fastest box "emerge -ep @world" takes about 3 seconds since 
there's almost no packages installed.
On my currently slowest box same takes about 15 minutes, because (1) lots of 
packages installed and (2) slow CPU and (3) brutally slow IO

Binpkgs only help so much - if any dependencies change I need to sync the 
configuration between client and server, and to see if it resolves I need to do 
a dryrun on the client (or be very certain that the configuration really 
matches) - or risk that it's going to fail because of mismatches.


I haven't quite figured out why portage needs such humongous amounts of memory 
(>300MB ?!) and why it needs to spend a minute to figure out how to install a 
simple almost-no-dependencies tool like htop or iotop. There's some obvious 
badness, and just saying "well let's use binpkgs" won't fix it (and, well, on 
the binpkg server if you have 3k packages installed you get the same 
performance issues, so ignoring the issue is kinda not a good idea)

There's been some good progress, I remember you reducing memory usage already 
(>500MB -> 350MB or so for merging kernel sources) and there's some speedups 
from the last round of performance work. Still I think we can do a lot better 
:)

Thanks for your efforts,

Patrick