Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-soc
Navigation:
Lists: gentoo-soc: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: gentoo-soc@g.o
From: mmacleod@...
Subject: Re: Improved binary package support
Date: Mon, 23 Mar 2009 08:52:38 +0200
> > I am interested in submitting a proposal that is loosely based on, but a
> > little larger in scope then, the "Improved binary package support" Idea
> > in the ideas list.
> > Which source repository should I submit a patch for if I want to do this
> > proposal?
>
> Please elaborate here. This mailing list is archived and will be
> available to others as they join the discussion via
> http://archives.gentoo.org
I have been interested in a long time with the idea of making Gentoo more 
binary friendly and have put quite a bit of thought into this idea over the 
last few years but have never had time to look into this fully.
Just recently I have for the first time in years had the time to look into 
something like this and have recently being visiting the idea again, this is 
something I might have tried to do even without gsoc, but gsoc makes it even 
more appealing to me.

I really like the flexibility that Gentoo use flags provide but this comes at 
a price. Almost everything must be compiled manually and a vast amount of time 
and resources are "wasted" as a result of this, this also turns many people 
away from Gentoo as they simply don't have the time to wait for compiles.
The compiling part is not a necessary side effect of the flexibility, however 
the hardware required to build/host a binary for every possible combination is 
simply way out of reach for now.

The really frustrating part of this is that a lot of this compiling is 
duplicated over and over again, the same package is compiled by many people 
with the exact same use flag and cflag settings.
The main part of my proposal then centers around taking advantage of this.
Below is a rather simplified version of what I would do, there are a lot of 
security and other issues that would need to be fleshed out as part of the 
project, I will go more into depth in my actual proposal.

A p2p network of willing Gentoo users could be created where packages compiled 
by people are shared, users could select to participate in various different 
forms(or combinations of these forms) by default only (1) would be used:
1/ Use packages from p2p network only.
2/ Donate packages created by us and not already on network to network. Or 
Hash signatures in cases where package is already on network.
3/ Donate bandwidth/harddrive space and act as a "mirror" on the network, 
packages would be distributed between mirrors so that maximum redundancy 
exists on all packages.
4/ Donate idle CPU time to act as a compile server, and compile packages 
combinations that are not yet on the network. Using an algorithm to cover more 
frequent cflag combinations first.

There obviously might be security concerns for users of such a system, this 
can be mitigated by using a central controlling server and by taking Hash 
signatures of compiled packages from users other then the original 
contributer. Different levels of trust can then be established for each 
package based on how many cross verifications of the Hash signature have been 
made. Users can then choose a level of trust and only use packages that meet 
this requirement.
Users who still consider this to large a risk can simply not enable the p2p 
binary support at all and continue to compile everything obviously.

A fairly large part of this idea is reliant on improving portages binary 
package support as stated in the idea on the wiki. Storing packages based on 
build state and use flags etc.. so all of this would also be done as part of 
the project. Therefore my proposal is basically to implement everything in the 
"improved binary package support" idea(Except the kernel part) and then to 
implement the rest of the system after that.


The above might sound a bit ambitious however I think for me that just the 
"Improved binary package support" is not nearly enough to fill up 3 months of 
work.
I have 3 1/2 years COTS software development experience(C++ mostly) as well as 
a bsc computer science and am busy finishing my honours degree part time, so I 
do have a good idea of the amount of work involved in such a system and 
already have good knowledge of the various tools etc.

I am willing to take a middle ground and have the "Improved binary package 
support" as the actual proposal and then the rest of the idea(p2p binary 
network) as an extended goal if this would make you more comfortable though.



Replies:
Re: Improved binary package support
-- Philipp Riegger
References:
Improved binary package support
-- mmacleod
Re: Improved binary package support
-- Jeremy Olexa
Navigation:
Lists: gentoo-soc: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: Improved binary package support
Next by thread:
Re: Improved binary package support
Previous by date:
Re: GLEP 25 + GLEP 9
Next by date:
Re: Interest in idea 2.16: SCM snapshot management infrastructure or 2.1: Add "tags" support to Portage


Updated Jun 17, 2009

Summary: Archive of the gentoo-soc mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.