Re: [gentoo-user] distcc and crossdev, anyone?

[Bill Longman]

On 12/18/2010 07:15 AM, Peter Humphrey wrote:
> On Saturday 18 December 2010 10:18:43 Neil Bothwick wrote:
>
>> I've found there's just too much overhead with distcc, plus much of
>> the work is still done locally.
>
> I expected that but I wanted to try it to see.
>
>> I have a couple of Atom boxes, a server and a netbook, and I've set up
>> a chroot for each on my workstation. In the chroot I have
>> FEATURES=buildpkg, using an NFS mounted PKGDIR available to both
>> computers, then I emerge -k on the Atom box.
>
> Maybe I'll go this way instead. Thanks for the idea, which is similar to
> one from YoYo Siska three days ago.

I had my Atom 330 running as a distcc client for a long time. I have
several other speedy CPUs alongside it so it could spray plenty of
compilation requests out its gigabit NIC to various much beefier
machines. But as Neil stated, lots of the processing still occurs
locally, so as you increase nodes, you need to decrease the amount of
compilation done locally. With such a disparity between CPU, it takes
less time overall to just do it the way Neil describes - make a chroot
and then just build it with the intention that the slow CPUs will use
the binary build.

You still need lots of CPU to compile, so a slow machine will still
compile slowly. If your client is a pokey 1.6GHz Atom and you're sending
jobs to two quad core 3GHz CPUs on your subnet, you'll soon see that the
Atom's load goes up toward 8 as it tries to bring those remote jobs
back. So, the four threads on my 330 get completely filled up and it's
dog slow. And it's even more painful when you use the preprocessor
because the client must zip the compile "construction" before it ships
it out, so you have even less CPU available for compilation (although
you get some of that back).

All said and done, my back-of-the-napkin and seat-of-the-pants
calculation tells me that I still get a _minimum_ 25% reduction in
overall compile times with distcc. That's my experience after using
distcc for almost ten years with various configurations of network and CPUs.