Re: [gentoo-cluster] High-Availability Howto for Gentoo

[Brady Catherman]

nono.. we where running using distcc across our cluster (you could  
see the compile jobs running on all the different nodes.)

Though I would not be surprised in the least if I configured  
something wrong.. Plus our Mac OS NFS Servers have been... flaky.. at  
best. So there might just be an issue in network file system  
performance.

It appears that my experience with distcc is wrong =) Thats good to  
know.. next time I have to re-build a cluster I will make sure to  
give it another go and see if I can make it perform better =)


On Apr 11, 2006, at 1:51 PM, Donnie Berkholz wrote:

> Brady Catherman wrote:
>> One last question for you all.. Why is distcc so popular? We used
>> it on
>> our 134 node cluster and it actually made compiling much slower than
>> just running it on one of the nodes. The network overhead killed the
>> performance gain. The only way we found that it helped was writing
>> the
>> makefile itself to take advantage of parallelism. Is this uncommon
>> for
>> most people?
>
> Autotools (properly used) create parallelizable Makefiles, so
> that's not
> much of an issue. Clearly if you're just exporting the same job to
> another node instead of parallelizing across multiple nodes, you will
> see a performance loss.
>
> distcc is particularly useful when not all nodes are attempting to
> locally upgrade/install something at the same time, so they team up
> on a
> parallelized compilation for a single node. There's no effective
> gain by
>  using distcc on a large cluster if you're just compiling
> everything on
> every node -- you should be using it in parallel to build binary
> packages once, then installing across all nodes.
>
> distcc is also useful when you've got a mixture of slow and fast
> nodes,
> for obvious reasons.
>
> Thanks,
> Donnie
>

-- 
gentoo-cluster@g.o mailing list