[gentoo-portage-dev] Re: portage-2.2-rc3 parallel merges quit being parallel

[Duncan]

Andrew Gaffney <agaffney@g.o> posted 488B9D84.4050209@g.o,
excerpted below, on  Sat, 26 Jul 2008 16:56:20 -0500:

> Duncan wrote:
>> "--jobs=10 --keep-going --load-average=15"
>
> For a dual-dual-core setup, a load average of 4.0 is "fully loaded".
> Anything higher than that and you're just causing jobs to queue up
> unnecessarily and your system to "thrash".

Not really.  The highest system load-average possible is the one-minute 
load-average, right?  From my experience there are times when it just 
sits there doing nothing.  No I/O, CPU graphs low, but load average still 
high so it won't start any more jobs.

I see gains at times up to ~4 jobs per core (tho it's arguably possible 
they're counteracted above say, 3/core, by extra shuffling, I won't argue 
that and haven't checked that closely, I just don't like to see blank 
spots in the CPU utilization that aren't accounted for by I/O).  I just 
boosted it to five so I could do the below....

>> have MAKEOPTS="-j -l20" so it's not going to be low all the time).
>
> Same thing here. Also, why would you specify different --load-average
> values in these two places?

The idea here is to create a differential, so it doesn't start new builds 
when a single build can adequately parallelize.  I'm building in tmpfs, 
which is (limited quantity, 8 gigs, but...) memory, and if a single 
emerge is keeping the system sufficiently busy, no reason to add a new 
one to the pile.  However, when a single merge isn't keeping the system 
busy, /then/ add more.  The differential at least in theory should favour 
single packages when they can provide sufficient parallelization. 

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman