Gentoo Archives: gentoo-portage-dev

From: Duncan <1i5t5.duncan@×××.net>
To: gentoo-portage-dev@l.g.o
Subject: [gentoo-portage-dev] Re: portage-2.2-rc3 parallel merges quit being parallel
Date: Sun, 27 Jul 2008 09:00:15
Message-Id: pan.2008.07.27.08.59.52@cox.net
In Reply to: Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel by Marius Mauch
1 Marius Mauch <genone@g.o> posted
2 20080727033210.f3441ab3.genone@g.o, excerpted below, on Sun, 27
3 Jul 2008 03:32:10 +0200:
4
5 > I did some benchmarking a while ago with different combinations of -j
6 > and -l in MAKEOPTS, using the kernel as testcase, and IIRC the fastest
7 > builds were around -l6.0 (on a dual-core system) and high (or unlimited)
8 > values for -j
9
10 FWIW... My experience suggests that there's some sort of race condition
11 with the job count. I was getting occasional very irritating errors to
12 the effect of "Job count is 17, should be 16" (or possibly the reverse),
13 that would terminate the build. Rerunning it wouldn't trigger the
14 problem again. By setting unlimited jobs (-j with no appended number), I
15 avoided whatever it was, or at least haven't seen it since.
16
17 I don't know enough about make's parallel job processing (OK, I simply
18 know that it normally works, which makes it irritating when it doesn't)
19 to have the foggiest what that was all about, except to assume it had to
20 be some sort of race condition on the job counter.
21
22 Has anyone else seen similar?
23
24 Anyway, that's why I ultimately ended up with -j -lX, using the -lX part
25 to do the limiting. With the previous simple job-at-a-time emerging, the
26 few cases where -lX wasn't honored and therefore wasn't limiting wasn't a
27 problem. Of course, there's some adjusting to be done now. =8^) But
28 it's for a good reason! =8^) (I've already decided that --jobs=10 is
29 going to need changed, I'm intuiting it needs to go up, but it's possible
30 I need to lower it. More experimentation is necessary! =8^)
31
32 --
33 Duncan - List replies preferred. No HTML msgs.
34 "Every nonfree program has a lord, a master --
35 and if you use the program, he is your master." Richard Stallman