1 |
Brady Catherman wrote: |
2 |
> One last question for you all.. Why is distcc so popular? We used it on |
3 |
> our 134 node cluster and it actually made compiling much slower than |
4 |
> just running it on one of the nodes. The network overhead killed the |
5 |
> performance gain. The only way we found that it helped was writing the |
6 |
> makefile itself to take advantage of parallelism. Is this uncommon for |
7 |
> most people? |
8 |
|
9 |
Autotools (properly used) create parallelizable Makefiles, so that's not |
10 |
much of an issue. Clearly if you're just exporting the same job to |
11 |
another node instead of parallelizing across multiple nodes, you will |
12 |
see a performance loss. |
13 |
|
14 |
distcc is particularly useful when not all nodes are attempting to |
15 |
locally upgrade/install something at the same time, so they team up on a |
16 |
parallelized compilation for a single node. There's no effective gain by |
17 |
using distcc on a large cluster if you're just compiling everything on |
18 |
every node -- you should be using it in parallel to build binary |
19 |
packages once, then installing across all nodes. |
20 |
|
21 |
distcc is also useful when you've got a mixture of slow and fast nodes, |
22 |
for obvious reasons. |
23 |
|
24 |
Thanks, |
25 |
Donnie |