Re: [gentoo-cluster] High-Availability Howto for Gentoo

[Donnie Berkholz]

Brady Catherman wrote:
> One last question for you all.. Why is distcc so popular? We used it on
> our 134 node cluster and it actually made compiling much slower than
> just running it on one of the nodes. The network overhead killed the
> performance gain. The only way we found that it helped was writing the
> makefile itself to take advantage of parallelism. Is this uncommon for
> most people?

Autotools (properly used) create parallelizable Makefiles, so that's not
much of an issue. Clearly if you're just exporting the same job to
another node instead of parallelizing across multiple nodes, you will
see a performance loss.

distcc is particularly useful when not all nodes are attempting to
locally upgrade/install something at the same time, so they team up on a
parallelized compilation for a single node. There's no effective gain by
 using distcc on a large cluster if you're just compiling everything on
every node -- you should be using it in parallel to build binary
packages once, then installing across all nodes.

distcc is also useful when you've got a mixture of slow and fast nodes,
for obvious reasons.

Thanks,
Donnie