Re: [gentoo-user] emerge -j, make -j and make -l

[Pandu Poluan]

On Nov 28, 2011 3:21 AM, "Michael Mol" <mikemol@×××××.com> wrote:
>
> On Sun, Nov 27, 2011 at 12:56 PM, Pandu Poluan <pandu@××××××.info> wrote:
> > On Nov 27, 2011 5:12 PM, "Michael Mol" <mikemol@×××××.com> wrote:
>
> [snip]
>
> >
> > Here's my experience:
> >
> > I always experience emerge failures on my Gentoo VMs if I use
MAKEOPTS=-j>3.
> > Not all packages, but many. Including, IIRC, glibc and gcc.
>
> In my barebones 177-package state, I didn't get any build failures
> from parallel building, either via emerge -j or make -j. I did get one
> failure when I went to install X that worked fine on the second
> attempt.
>

And in my barebones system (172 packages), there are several packages that
reliably fail, even when emerged on their own *sigh*

> > This happens even if I make sure that there's just one emerge job being
> > done. And this happens even if I allocate more vCPUs than -j, on VMware
and
> > XenServer alike.
>
> FWIW, I've been running with MAKEOPTS=-j10 on my Phenom 9650 for over
> a year. It's very rare that something breaks due to the parallel
> build. I think it's happened perhaps three times, and each time was
> resolvable with a retry. YMMV, of course; race conditions are finicky.
>

I have a hunch that the hypervisor had some effects. Most likely, another
VM in the same host has a high enough load that necessitates the Gentoo VM
to be shifted to another (physical) core, and this messes up the timing.

MAKEOPTS=-j3 always succeeds, though. So I'm disinclined to delve deeper
into the issue.

> > I don't know where the 'blame' lies, but I've found myself
standardizing on
> > MAKEOPTS=-j3, and PORTAGE_DEFAULT_OPTS="--jobs
> > --load-average=<1.6*num_of_vCPU>"
> >
> > (Yes, no explicit number of jobs. The newer portages are smart enough to
> > keep starting new jobs until the load number is reached)
>
> Sweet; I didn't know about Portage's --load-average; I'll definitely
> switch to that instead of -j. Load-driven make plus load-driven
> portage should work beautifully on my system.
>
> I'll steal your 1.6 factor, and give:
>    MAKEOPTS=-j <2*N> -l <1.6*N)
>    PORTAGE_DEFAULT_OPTS="--jobs --load-average<1.6*N>"
> a try.
>

Make sure that make supports non-integer values for -l, though.

> How does that interact with distcc, by the way? I've got two of these
> octo-core Xeon boxes, and I've still got my Phenom 9650--distcc on my
> home network should become very, very nice. And this box is consuming
> 255W at the wall with monitor off, 326W with monitor on. That's not
> bad. Though perhaps I should move to an apartment where heat isn't
> free...
>

Well, I don't use distcc, so I can't speculate. But with plain single
system compilation, my settings have been serving me real well :-)

Rgds,