[gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

[Duncan]

So I'm running the 2.2-rcs and have been seeing blogs about the new 
parallel merge capacities...  Having a dual-dual-core Opteron and having 
run multiple merges manually for some time, this is VERY welcome news. 
=8^)

So after upgrading to -rc3 I set EMERGE_DEFAULT_OPTS to include

"--jobs=10 --keep-going --load-average=15"

and tried a few small merges (the rest of the world update).  It worked 
nicely.

So then, as I had adjusted LDFLAGS recently and hadn't yet done a full 
world remerge, I decided to try the BIG test, emerge --emptytree --world, 
with some 674 packages.

For the first 100 or so packages, it worked quite well.  However, about 
there, maybe package 120 or so, so about 20% of the way thru, it reverted 
to doing them one-at-a-time again.  I'm now on package #279 and it's 
still doing them only one at a time, and this has included stuff like the 
xorg-docs (IIRC) package, that really shouldn't be pre-deps for stuff 
that has come since, so /something/ else should have been trying to 
merge, as long as load-average stays low, as it has much of the time (I 
have MAKEOPTS="-j -l20" so it's not going to be low all the time).

Is this a known issue still being worked on, perhaps due to the limits of 
the package dependency and scheduling resolution such that it can't 
handle a full world remerge and defaults back to one-at-a-time after it 
reaches the extent of its mapping, or is this a new bug?

Also, a little monitoring utility that could be run in another terminal 
and just list and update all the currently merging packages, and any that 
had failed to merge, so I could take a look at them while the system is 
still working on the rest, would be quite useful.  I tried watch ls -d 
$PORTAGE_TMPDIR/*/* and it starts to work, but of course starts to error 
in a few seconds when the first package completes since the *s are 
resolved initially.  With a bit of work I'm sure can get something simple 
working, but it'd be nice if there were a pre-made utility to do it.  
Maybe there is and I just don't know about it yet?  =8^)

Finally... I was rather confused the first time at just one job an 
install took a bit, as that's apparently not counted as "running", so it 
appeared nothing was going on for a bit.  Maybe an "installing" count as 
well would be useful... and prevent that confusion.

Thanks, guys.  It's already fun playing with! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman