Re: [gentoo-user] Jobs and load-average

[Michael]

On Wednesday, 15 February 2023 14:31:25 GMT Peter Humphrey wrote:
> On Wednesday, 15 February 2023 13:18:24 GMT Rich Freeman wrote:
> > On Wed, Feb 15, 2023 at 4:56 AM Peter Humphrey <peter@××××××××××××.uk>
>
> wrote:
> > > Not long ago I read that we should allow 2GB RAM for every emerge job -
> > > that is, we should divide our RAM size by 2 to get the maximum number of
> > > simultaneous jobs. I'm trying to get that right, but I'm not there yet.
> > >
> > > I have these entries in make.conf:
> > > EMERGE_DEFAULT_OPTS="--jobs=16 --load-average=32 --autounmask=n --quiet-
> > > unmerge-warn --ke>
> > > MAKEOPTS="-j16"
> > >
> > > Today, though, I saw load averages going up to 72. Can anyone suggest
> > > better values to suit my 24 threads and 64GB RAM?
> >
> > First, keep in mind that --jobs=16 + -j16 can result in up to 256
> > (16*16) tasks running at once.  Of course, that is worst case and most
> > of the time you'll have way less than that.
> >
> > Keep in mind that you need to consider available RAM and not just
> > total RAM.  Run free under the conditions where you typically run
> > emerge and see how much available memory it displays.  Depending on
> > what you have running it could be much lower than 64GB.
> >
> > Beyond that, unfortunately this is hard to deal with beyond just
> > figuring out what needs more RAM and making exceptions in package.env.
> >
> > Also, RAM pressure could also come from the build directory if it is
> > on tmpfs, which of course many of us use.
> >
> > Some packages that I build with either a greatly reduced -j setting or
> > a non-tmpfs build directory are:
> > sys-cluster/ceph
> > dev-python/scipy
> > dev-python/pandas
> > app-office/calligra
> > net-libs/nodejs
> > dev-qt/qtwebengine
> > dev-qt/qtwebkit
> > dev-lang/spidermonkey
> > www-client/chromium
> > app-office/libreoffice
> > sys-devel/llvm
> > dev-lang/rust   (I use the rust binary these days as this has gotten
> > really out of hand)
> > x11-libs/gtk+
> >
> > These are just packages I've had issues with at some point, and it is
> > possible that some of these packages no longer use as much memory
> > today.
>
> Thank you all. I can see what I'm doing better now. (Politicians aren't the
> only ones who can be ambiguous!)
>
> I'll start by picking up the point I'd missed - putting MAKEOPTS in
> package.env.

You can have both a generic MAKEOPTS in make.conf, which suits your base case 
of emerge operations and will not cause your PC to explode when combined with 
EMERGE_DEFAULT_OPTS, as well as package specific MAKEOPTS in package.env to 
finely tune individual package requirements.