1 |
Resending to portage dev ml to solicit comments... |
2 |
|
3 |
Thoughts welcome, since it's an issue that must be dealt with to |
4 |
sanely do efficient parallelization (monkeying with make.conf and |
5 |
hardcoding vals isn't much of a solution). |
6 |
|
7 |
Zac- comments? You seem to have totally missed the original email ;) |
8 |
|
9 |
------------------------------------- |
10 |
Date: Sun, 1 Oct 2006 12:27:13 -0700 |
11 |
To: gentoo-dev@l.g.o |
12 |
From: Brian Harring <ferringb@×××××.com> |
13 |
Subject: Re: [gentoo-dev] Setting number of parallel builds for other build-systems than 'make' |
14 |
|
15 |
On Sun, Oct 01, 2006 at 09:52:14AM -0700, Donnie Berkholz wrote: |
16 |
> Tiziano Müller wrote: |
17 |
> >-----BEGIN PGP SIGNED MESSAGE----- |
18 |
> >Hash: SHA1 |
19 |
> > |
20 |
> >Hi everyone |
21 |
> > |
22 |
> >It seems that setting the number of parallel builds using '-jN' does not |
23 |
> >only work for make, but also for scons and bjam (and maybe others as well). |
24 |
> >Since it isn't save to assume that '-jN' is the only option in MAKEOPTS, |
25 |
> >some filtering is needed. |
26 |
> >Now, SCONSOPTS (BJAMOPTS respectively) could be added to make.conf and used |
27 |
> >whenever one of those build-systems is being used. But we would probably |
28 |
> >have to add a ...OPTS for every build-system. |
29 |
> > |
30 |
> >What do you think? Better ideas? |
31 |
> |
32 |
> I think adding it for each build system is probably a good idea, |
33 |
> nobody's guaranteeing option-level compatibility with make. Optionally |
34 |
> falling back to using the few valid MAKEOPTS might be a nice addition. |
35 |
|
36 |
I might be daft (likely), but why not just introduce a var indicating |
37 |
max parallelization instead? Tweak portage to push that setting into |
38 |
MAKEOPTS="${MAKEOPTS+${MAKEOPTS} } -j${PARALLELIZATION}". |
39 |
|
40 |
Might sound daft, but -j is hardcoded parallelization; if you're |
41 |
trying to run 3 build processes, the original var of -j isn't all that |
42 |
useful, nor will the hardcoded -j# var set for 3 package build |
43 |
processes be useful if you're doing a single build. |
44 |
|
45 |
Depending on number of seperate package build processes possible, the |
46 |
# of build processes allocated per build process needs to vary |
47 |
(essentially). |
48 |
|
49 |
Now... that's a bit ahead of what portage is doing atm, but folks |
50 |
*will* parallelize portage building (see bug 147516 if in doubt), so |
51 |
its not too far out. |
52 |
|
53 |
Getting back to the actual topic, set this var, it's a raw int that a |
54 |
scons/bjam eclass can use to easily set appropriate var with the |
55 |
parallelization requested, thus unifying this particular knob; more |
56 |
importantly, it gives them a way to do what they're after while |
57 |
exposing a knob that the pkg manager can easily fiddle with for global |
58 |
parallelization needs. |
59 |
|
60 |
Only downside to it I see is that it requires mangling the pkg manager |
61 |
to translate the parallelization setting into MAKEOPTS+=-j#; can't |
62 |
really get around that however due to the fact MAKEOPTS is already |
63 |
forced in installed configuration though. |
64 |
|
65 |
Thoughts? |
66 |
~harring |