Re: [gentoo-soc] Project Proposal : GenCC for "Gentoo Community Compiling"

[Rich Freeman]

On Mon, Feb 18, 2013 at 9:03 AM, Patrick Lauer <patrick@g.o> wrote:

(note, switching order I comment in)

(Note also - I'm not sold on any of this being practical - this is an
outline of how the obstacles might be surmounted.)

>
> How do you manage to get a precise environment (including useflags,
> correct gcc version, ...) onto the compile host? Wouldn't that be slower
> than building it locally?

So, the premise is that if you build a package with identical
dependencies you'll get an identical output.  That is using a
definition of dependency as anything involved in the creation of the
package.

So, for this to work you'd need to:
1.  ID all the dependencies of a package (INCLUDING system packages).
We don't do this for various reasons, and that would be a problem,
unless the data is somehow gathered by the tool or we change our
policies.

2.  ID the versions and configurations of all the dependencies.  That
would include USE flags at least.  Hopefully nothing build-time
depends on configuration files, but that might also be an issue.

3.  ID hosts who have identical configurations, and use them to build.

However, if you're going to do all of that stuff, you could just as
easily assemble a binary packages catalog.  Why distribute all the
compiling when you can just ask for what should be an identical final
executable?

> So, how do you handle people being evil? What happens if my genccd
> always returns 0-byte files? what if it adds random things to execute
> code on the user's system?

I'd also want to make sure that the attacks can't go the other way as
well.  If nothing else you're open to a CPU-using DOS (compile this
2GB file full of really complex C++ abstractions for me).

For the output you could have some kind of web-of-trust where multiple
people do the work and report the hash/etc.

Again, a binary package repository shouldn't really be any harder to
implement and makes far more sense all around.  Then everybody just
reports their hashes, or we could have an official list of these and
mirrors/etc.

Distributed CC within an organization makes more sense as it
eliminates the trust issues, mostly (you might still care about
reliability, but not malicious attacks).  However, a single
organization is also likely to have more uniform configuration, and
thus a binary repository makes even more sense.

I think the big question is - what does distributed compilation get us
that a binary repository doesn't get us, or in what way is it easier?
I can't really think of any advantages of the former over the latter.
Both will still be a challenge to implement, and that might start with
fully identifying dependencies.

Rich