1 |
nono.. we where running using distcc across our cluster (you could |
2 |
see the compile jobs running on all the different nodes.) |
3 |
|
4 |
Though I would not be surprised in the least if I configured |
5 |
something wrong.. Plus our Mac OS NFS Servers have been... flaky.. at |
6 |
best. So there might just be an issue in network file system |
7 |
performance. |
8 |
|
9 |
It appears that my experience with distcc is wrong =) Thats good to |
10 |
know.. next time I have to re-build a cluster I will make sure to |
11 |
give it another go and see if I can make it perform better =) |
12 |
|
13 |
|
14 |
On Apr 11, 2006, at 1:51 PM, Donnie Berkholz wrote: |
15 |
|
16 |
> Brady Catherman wrote: |
17 |
>> One last question for you all.. Why is distcc so popular? We used |
18 |
>> it on |
19 |
>> our 134 node cluster and it actually made compiling much slower than |
20 |
>> just running it on one of the nodes. The network overhead killed the |
21 |
>> performance gain. The only way we found that it helped was writing |
22 |
>> the |
23 |
>> makefile itself to take advantage of parallelism. Is this uncommon |
24 |
>> for |
25 |
>> most people? |
26 |
> |
27 |
> Autotools (properly used) create parallelizable Makefiles, so |
28 |
> that's not |
29 |
> much of an issue. Clearly if you're just exporting the same job to |
30 |
> another node instead of parallelizing across multiple nodes, you will |
31 |
> see a performance loss. |
32 |
> |
33 |
> distcc is particularly useful when not all nodes are attempting to |
34 |
> locally upgrade/install something at the same time, so they team up |
35 |
> on a |
36 |
> parallelized compilation for a single node. There's no effective |
37 |
> gain by |
38 |
> using distcc on a large cluster if you're just compiling |
39 |
> everything on |
40 |
> every node -- you should be using it in parallel to build binary |
41 |
> packages once, then installing across all nodes. |
42 |
> |
43 |
> distcc is also useful when you've got a mixture of slow and fast |
44 |
> nodes, |
45 |
> for obvious reasons. |
46 |
> |
47 |
> Thanks, |
48 |
> Donnie |
49 |
> |
50 |
|
51 |
-- |
52 |
gentoo-cluster@g.o mailing list |