Gentoo Archives: gentoo-dev

From: Sam James <sam@g.o>
To: gentoo-dev@l.g.o
Cc: qa@g.o
Subject: Re: [gentoo-dev] [PATCH] check-reqs.eclass: clamp MAKEOPTS for memory/RAM usage
Date: Wed, 05 Jan 2022 20:20:07
Message-Id: 7F08DDB4-F77C-4920-968C-AF8B7B64B050@gentoo.org
In Reply to: Re: [gentoo-dev] [PATCH] check-reqs.eclass: clamp MAKEOPTS for memory/RAM usage by Ulrich Mueller
1 > On 5 Jan 2022, at 08:28, Ulrich Mueller <ulm@g.o> wrote:
2 >
3 >>>>>> On Tue, 04 Jan 2022, Sam James wrote:
4 >
5 >> Crank down MAKEOPTS jobs if MAKEOPTS="-jN" is too high for the
6 >> amount of RAM available (uses amount declared as needed
7 >> in the ebuild). Typically should be ~2GB per job.
8 >
9 > Where does this number 2 GB come from? The amount of RAM strongly
10 > depends on the programming language and other factors, so I don't
11 > believe that there's one number that can be used for everything.
12 > (If only considering C and C++ 2 GB seems to be excessive, i.e. it
13 > will limit parallel build more than necessary. When linking, the number
14 > may be _much_ larger.)
15 >
16
17 This is essentially "common law" (or "common lore" if you like!) and
18 is well-accepted as a good rule of thumb.
19
20 The number being larger doesn't really make any difference here;
21 the point is to help in cases where we're pretty sure there's going
22 to be a problem (only users of check-reqs.eclass, and we can
23 Introduce a variable to give ~RAM per job too if needed).
24
25 > Also not sure if I understand the arithmetic. Shouldn't it use
26 > CHECKREQS_MEMORY as the basis of the calculation?
27
28 As noted, I was sort of on the fence about whether to hardcode
29 or use the CHECKREQS_MEMORY value and I'll think about this
30 again. I went for the current approach partly to keep things simple
31 (to avoid needing min/max bits).
32
33 >
34 >> +# @ECLASS-VARIABLE: CHECKREQS_MEMORY_MANGLE_JOBS
35 >> +# @USER_VARIABLE
36 >> +# @DESCRIPTION:
37 >> +# Allow packages to reduce the number of multiprocessing (e.g. make, ninja) jobs
38 >> +# to lower memory usage.
39 >> +: ${CHECKREQS_MEMORY_MANGLE_JOBS=yes}
40 >
41 > If anything, the feature should be opt-in rather than opt-out.
42
43 That would defeat the purpose. If it's opt-in, then people are already aware
44 of their settings being inappropriate, and should probably fix them.
45
46 This is also about reducing the number of support queries about
47 builds which failed for trivial reasons, as someone who has to handle
48 quite a lot of those (until such a time as we can implement better
49 detection, but this is also a generally nice UX improvement -- as
50 per what Florian/flow said).

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies