Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs

[Eric Martin]

Stroller wrote:
>
> On 31 Jul 2008, at 01:59, Simon wrote:
>> ...
>>> Your email is very long, so I'm not sure if I've taken it all in, but
>>> what I'd suggest is a shared /usr/portage directory (easily done by
>>> NFS) and distcc.
>>
>> This creates a dependency on the host that contains the portage tree.
>> It also remove some flexibility. For example, taking my laptop away
>> from my 'portage pc' would make it impossible for me to modify my
>> current installs while away.  There are other consideration and the
>> use of a sync'ed portage tree on each pc vs using nfs is a debate that
>> I won't go into now and this part is not much of an importance in my
>> problem... compilation is!...
>
> Yes, your words about "dependency" and "flexibility" are valid, but this
> is also the most straightforward way to sync multiple machines at once.
> If you do need to emerge a package when the laptop is away from home
> then just --sync and it builds a portage tree at the "missing
> mountpoint" (if that makes sense).
>
> You may find it more elegant to make the one machine an rsync server for
> the others.
> http://www.gentoo.org/doc/en/rsync.xml
>
>>> distcc is, IMO, a bit more elegant than (for instance) trying to
>>> manually emerge binary packages for machine A on PC B. You can tell
>>> it to share the work or just unload it to the most powerful machine.
>>> There may be concerns about using a binary package if USE flags are
>>> different between the two machines, but distcc ensures that the
>>> package is built using those defined in make.conf of the machine on
>>> which you're running emerge.
>>
>> Yes!  I was actually trying distcc today for the first time and got it
>> working from the perspective of my fastest computer, I got some
>> trouble though (see below).  What you mentioned about running the
>> `emerge -uDN world` on each individual machines + sharing built
>> packages is absolutely awesome. Best of all worlds if i could say!
>
> Great! I'm glad you're happy with this. You're NFS exporting a
> sub-directory of /usr/portage, then, in order to share the built packages?
>
>> However, when using distcc, I first made a trial with a small package
>> 'xmahjongg' and got a nice x4 speedup on the overall emerge.  I wanted
>> to try with a larger package, 'povray' and stumbled on a linker issue,
>> the issue is described below and this is the only obstacle on my way
>> now.  As I fear doing a `emerge -e system && emerge -e world` would
>> never complete using distcc...
>>
>> Doing: `time emerge povray` without distcc yields a functionnal
>> package, while when distcc was enabled, I would get lots of undefined
>> references to some __pthreads functions.  But I just tried and it
>> seems to work fine, not reproducible, so I'll drop my distcc issue and
>> go on with the -e recompilation.
>
>
> I assume that the "undefined references to __pthreads function" are
> errors which stop the compile? Or that they occur when you start the
> app, causing it to crash? Rather than compilation warnings?
>
> I assume compilation errors. My usage is that I can turn distcc off for
> the duration of the compile when I see something like this, and not
> bother investigating it further, but I think the most likely cause is
> that a library is needed for compilation that is not present on the
> distcc server. Portage accepts the compile-time dependency because it is
> filled on the distcc client, the machine on which you've run emerge, but
> when that particular bit is   sent off to the distcc server then that
> machine doesn't have the lib needed.
>
> I would imagine that, assuming the above belief is correct, then the
> workaround would be to `emerge -o` the package on the other machines on
> your LAN (or the fastest machine, if you are using only that to emerge)
> before distcc'ing it. This is slightly inelegant.
>
> If you mostly have the same packages on all machines then hopefully you
> shouldn't encounter this scenario too often, although I'd also think
> that different USE flags could affect it.
>
> I'm also somewhat suspicious of different architectures - you wouldn't
> try compiling for ARM or MIPS on an x86 PC, but I'm not sure how
> compiling on an Athlon for a Pentium 3 or 4 affects things. Finally you
> should make sure all machines are using the same versions of gcc and
> glibc (also binutils? what else?).
>
> Stroller.
>

You can cross compile, I've done it multiple times before When my Athlon 
built packages for my celerons.  Check out the Distcc-Cross Compile 
guide here [1].  Apparently the doc has been updated since I last 
checked as the tools have gotten full rewrites.

1) http://www.gentoo.org/doc/en/cross-compiling-distcc.xml

-- 
Eric Martin
Key fingerprint = D1C4 086E DBB5 C18E 6FDA  B215 6A25 7174 A941 3B9F