Re: [gentoo-portage-dev] Max parallelization setting

[Francesco Riosa]

Brian Harring ha scritto:
> On Tue, Oct 10, 2006 at 05:27:07PM +0200, Marius Mauch wrote:
>
>> On Tue, 10 Oct 2006 00:04:57 -0700
>> Brian Harring <ferringb@×××××.com> wrote:
>>
>>
>>> I might be daft (likely), but why not just introduce a var indicating
>>> max parallelization instead?  Tweak portage to push that setting into
>>> MAKEOPTS="${MAKEOPTS+${MAKEOPTS} } -j${PARALLELIZATION}".
>>>
>>> Might sound daft, but -j is hardcoded parallelization; if you're
>>> trying to run 3 build processes, the original var of -j isn't all
>>> that useful, nor will the hardcoded -j# var set for 3 package build
>>> processes be useful if you're doing a single build.
>>>
>>> Depending on number of seperate package build processes possible, the
>>> # of build processes allocated per build process needs to vary
>>> (essentially).
>>>
>> Seems useful as *alternative* to the low level vars, but shouldn't
>> replace them (so if the low level vars are set they override the global
>> setting).
>>
>
> Well.. heres the failing.
>
> Say I've got a 32x system; I screw up and have -j32 in my makeopts,
> and PARALLELIZATION=32.
>
> Now either the pkg manager needs to understand MAKEOPTS (beyond just
> adding a simple string to it), and check for -j*, or it's going to
> wind up allowing 32 seperate build jobs, each with -j32.
>
> Personally, I *really* would love to see that loadavg (that would be
> one nifty screenshot).
>
> Allowing MAKEOPTS to override the alloted build slices  the manager
> hands it totally defeats the purpose of moving the parallelization
> factor into a seperate var; the intention is to give the manager a
> max, and allow it to allocate as it needs depending on the # of build
> jobs that can be ran at that point.
>
> Theres no point in doing this if the hole it's trying to plug is
> explicitly allowed; at best you wind up with two different vars you
> have to keep in sync, at worst, you wind up with the 32*32 exapmle
> from above.
>
> Response of course is "don't have -j* in your MAKEOPTS"; well dar,
> thus why allow it? ;)
>
> ~harring
>
Not going deep, sincerly I've read too few form the bug and the thread but:
I'm lucky and I have a cluster of 32 boxes with distcc each with 32
processors.
In such a eco-system the ebuild work is not parallelizable in the
cluster but the compiler work is, those are very different things.
please keep this in account ;)

rgds,
Francesco R.
-- 
gentoo-portage-dev@g.o mailing list