Gentoo Archives: gentoo-dev

From: Philippe Trottier <tchiwam@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] Parallizing ebuilds - 'trivial' ebuilds
Date: Fri, 13 Jan 2006 19:07:50
Message-Id: 43C7F9EC.6080009@gentoo.org
In Reply to: Re: [gentoo-dev] Parallizing ebuilds - 'trivial' ebuilds by Kalin KOZHUHAROV
1 -----BEGIN PGP SIGNED MESSAGE-----
2 Hash: SHA1
3
4 Kalin KOZHUHAROV wrote:
5 > Patrick Lauer wrote:
6 >> On Fri, 2006-01-13 at 19:53 +0900, Kalin KOZHUHAROV wrote:
7 >>
8 >>>> Make this distributed tool for tar zip bzip2 and gzip and I'm in, I
9 >>>> don't think it would be useful with anything else than Gigabit Ethernet.
10 >> One 2Ghz CPU can't even saturate a 100Mbit line with bzip2 as far as I
11 >> can tell.
12 >> Although the speedups won't be extreme it could just work.
13 >>
14 >>
15 >>>> We might want to have in the make.conf 2 separate variables, one of them
16 >>>> saying how many threads can be run on the machine, then How many
17 >>>> threads/process across a cluster.
18 >>>>
19 >>>> For example, my Dual Xeon EM64T file server can do make -j4 locally,
20 >>>> like in make install, make docs etc etc, But for compiling I can use
21 >>>> -j20, really not useful over -j8 anyway. But the point is, it would be
22 >>>> usefully to separate the load distribution on the local machine and
23 >>>> cluster nodes.
24 >>> As the discusison started...
25 >>>
26 >>> I would like to be able to limit the -jN when there is no distcc host
27 >>> available or when compiling c++ code, otherwise my poor laptop is dead with
28 >>> -j5 compiling pwlib when the network is down....
29 >> As far as I can tell distcc isn't smart enough for dynamic load balancing.
30 >> One could hack portage to "test" each server in the distcc host list and
31 >> remove missing servers for each run - doesn't look elegant to me.
32 >
33 > Yes, might be a solution, even if not elegant. I am thinking also of
34 > automating distcc configuration (i.e. no need to run --set-hosts) and one
35 > idea is to use DNS with some TXT record, but that is just an idea - no
36 > patching is done yet.
37
38 Recipe for disaster, specially in a place like mine where sparc, alpha, x86_64
39 and ppc32/64 mix... not counting ia64 for a test run soon...
40
41 If you really want to do this, someone has to make a rendezvous a la Apple.
42 Where not only distcc says I am available but I am also doing the right stuff.
43
44
45
46 -----BEGIN PGP SIGNATURE-----
47 Version: GnuPG v1.4.2 (GNU/Linux)
48 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
49
50 iD8DBQFDx/nsP0/FkJ0eBc0RAn8aAJ9h8C1QFO0HyQmBINl2erPljgaEYwCfcOBF
51 yPpDJ/KElWfotWqNvWYPq3s=
52 =XM29
53 -----END PGP SIGNATURE-----
54 --
55 gentoo-dev@g.o mailing list

Replies

Subject Author
Re: [gentoo-dev] Parallizing ebuilds - 'trivial' ebuilds Philipp Riegger <lists@××××××××××××.de>