Gentoo Archives: gentoo-user

From: YoYo Siska <yoyo@××××××.sk>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Re: Simultaneously emerging multiple packages with same dependencies
Date: Thu, 27 Jan 2011 18:14:54
Message-Id: 20110127181325.GA26616@ksp.sk
In Reply to: Re: [gentoo-user] Re: Simultaneously emerging multiple packages with same dependencies by "J. Roeleveld"
1 On Thu, Jan 27, 2011 at 03:12:49PM +0100, J. Roeleveld wrote:
2 > On Thursday 27 January 2011 15:05:25 YoYo Siska wrote:
3 > > On Thu, Jan 27, 2011 at 03:33:21PM +0200, Nikos Chantziaras wrote:
4 > > > On 01/27/2011 03:11 PM, Dale wrote:
5 > > > >[...]
6 > > > >I am using the -j option for the first time now. I'm updating KDE. It
7 > > > >seems to work fine. It doesn't scroll all the stuff like with a regular
8 > > > >emerges but this new rig is so fast, I can't read it anyway. I did have
9 > > > >a package to fail and it spit out the error for me to read.
10 > > >
11 > > > You don't need that if you have MAKEOPTS set in your make.conf,
12 > > > which is preferred. The -j option of emerge emerges multiple
13 > > > packages, while with MAKEOPTS set to "-j4" or whatever does a
14 > > > parallel build in the same package (meaning compiling multiple
15 > > > source files at the same time).
16 > > >
17 > > > It's preferred because with "emerge -jN" the last package will only
18 > > > use one CPU, while with "-jN" in MAKEOPTS even the last package will
19 > > > use N CPUs. Furthermore, emerge can't always build N packages at
20 > > > the same time because one can depend on the other, so it will have
21 > > > to wait until the dependency is built.
22 > >
23 > > On the other hand, unpacking, configure and install stages are not
24 > > parallel and emerge can do those in parallel for different packages...
25 > > The best would be somewhere in the middle ;)
26 > >
27 > >
28 > > There are also the load-average options to -j, i.e.:
29 > > MAKEOPTS="-j -l5" emerge -j --load-average=5 ....
30 > >
31 > > which makes make spawn parallel processes while load average is below 5
32 > > and the same for emerge spawning parallel ebuilds (when make isn't
33 > > parallel enough)
34 > >
35 > > yoyo
36 >
37 > Hmmm... didn't know about that one yet.
38 > Does that mean that by doing it like that, the emerge-process (and compile-
39 > processes) will try to keep the load average at 5 and if that is lower, it
40 > will keep adding more processes?
41 >
42
43
44 Yes.
45 It might not be perfect, but mostly it works pretty well.
46 Once make started 10 or so process, which ate all my ram, because I
47 forgot to reenable swap, when I was playing with something before that
48 :)
49
50 yoyo

Replies