[gentoo-dev] binary packages, crazy idea

[J Robert Ray]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, I have an idea I've been thinking about, I'd like to share it and
see anyone thinks there is any merit to it.

The goal is to preserve gentoo's customizability but gain the
convenience of instantly available binary packages.

The problem with binary packages is that they come in one flavor and
would ignore your local USE variable settings.  Plus, creating separate
binary packages for every combination of USE variables would be an
enormous, never ending task.

My idea is to automate the process of creating binary packages, and turn
the installed base of gentoo users into a giant compile farm.

When a user goes to emerge a package, first portage would analyze the
ebuild to determine what USE variables are used by the package.  Then it
would build an ordered list of the user's USE variables, limited to only
those that are used by this particular package, but indicating if they
are enabled or not.

To this list of USE variables would be added the gcc version, CHOST,
CFLAGS, CXXFLAGS, perhaps portage version, plus what ever else that
makes up the uniqueness of the user's configuration.  I'm taking a cue
from ccache for this.

Portage could then take this unique code and check a public server to
see if a binary package exists matching this combination.  If so, the
binary package would be fetched and installed.  If not, the package
would be compiled locally and then uploaded to the public server.

To prevent from malicious tampering, some kind of system would need to
be in place to verify that a package hasn't been trojaned or something.
~ The server could wait until it received a certain number of copies of a
specific package from unique sources and compare them to see if they are
all the same before deciding a package is safe.

Security would be the largest stumbling block of this system.  I think
the most likely system to be safe would be a PGP-like web of trust.
Each gentoo installation would generate a set of keys that it would use
to sign packages it uploads.  Small groups of gentoo users who have the
same setup could exchange keys.   As the server receives binary packages
with the same checksum, it accumulates the keys from the signature.
Portage would only then download a package if it has been signed by a
key that the user trusts.  Of course, if no such key is available, it
would build the package locally.  If the local binary package matches
what is available on the server, this user would sign the binary package
available on the server.  Optionally, a user to configure his system to
accept a binary package after it has been signed by a certain number of
keys.

I believe the ideal storage facility for such a system is with something
like freenet.  It is distributed, non-permanent storage where more
popular files are longer-lived than infrequently requested files.
Assuming the vast majority of gentoo users are using the same default
settings for most things, the popular packages would survive
indefinately on freenet as long as there are people installing them.

A system that took advantage of freenet could upload the binary package
to freenet with its md5sum as the key, and submit the package details
with key to a central server.  Portage would when check with the central
server for a key, fetch the package out of freenet with the key, and
then install the binary package.

That's basically it.  Sure, it's a far-fetched idea but I'd to hear what
people think about it.

- - Robert

- --
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBB929E54
Key fingerprint = BEA9 490C D2B9 AD83 E88B  3148 3136 34E4 BB92 9E54
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE9b3XqMTY05LuSnlQRAvMWAJwOqQQsfZiJOO9G8TpcD1EOLq9yTACeI8af
KGbprUFS/QwGRLwygQ0lA44=
=FxhM
-----END PGP SIGNATURE-----