Re: [gentoo-dev] GRP set consolidation

[Pieter Van den Abeele]

On 22 May 2004, at 00:09, Stuart Herbert wrote:

> On Friday 21 May 2004 23:11, Chris Gianelloni wrote:
>> The other take on this is gentoopix and what it will provide.
>
> What is gentoopix?  Does it have a GLEP number? ;-)

I didn't commit a GLEP for it, but I described and implemented the idea 
I think Chris is talking about for the livecd scripts I wrote and used 
for all ppc/g3/g4/g5 releases until Catalyst was put into production 
(My scripts available on http://www.metadistribution.org/gentoo/ and in 
gentoo cvs). The idea is the following:

- stage1 is a set of packages which are build on an existing system 
(doesn't have to be gentoo).
- stage2 is stage1 + extra packages
- stage3 is stage2 + extra packages
- grp is stage3 + extra packages
- livecd is grp + extra packages which make the system ( which consists 
of all the unpacked packages ) bootable as a livecd.

If my script is given a stage2 and asked to build a livecd, it will 
upgrade to stage3, then build grp and add a few packages to make the 
produced set of GRP bootable. For Historical reasons, Gentoo release 
tools (even current catalyst) are not following this design/idea: If I 
give catalyst a stage2 or a stage3 , and ask it to produce a set of GRP 
it will do so, but if I ask catalyst afterwards to build a livecd, it 
will just start from the stage2/stage3 again and compile kde/gnome (in 
case of the kde/gnome livecd), all over again. Such a thing leads to 
countless wasted hours for the engineers building our releases (incl. 
me). I have been working closely together with zhen, our current 
catalyst maintainer on the design of the tool, my remarks can be found 
in the REMARKS file in the catalyst cvsroot.

My original proposal was to just give the people a (kde/gnome, minimal, 
developer, game, whatever) livecd and then use a tool which 'derives' 
packages from the live environment on that cd, using the gentoo package 
manager knowledge. You just build one bootable, optimized livecd (with 
a compressed loop filesystem, you're able to put a +2G system on a 
regular 700M CD) for each of your subarchs (or just one generic), and 
when the user has booted into this system, he/she can create stages, 
GRP, or even a new livecd, using the packages derived from the livecd. 
For those packages which cannot be derived, portage can use it's 
knowledge (ebuilds) to build/compile/create the package. It will have 
to use its knowledge anyway if you're changing the CFLAGS, USE, 
whatever...

This system makes stages unnecessary for distribution: A stage3 
contains a stage2, so, from a mirror perspective, it makes sense to 
just have the packages, or the knowledge how to derive them, and have 
'virtual stages', or better 'metastages', declaring what packages a 
configuration consists of (our livecds run the stages they distribute 
anyway):

- a desktop stage, a server stage, an xfree stage, the original 
stage2...

In theory we could include milions of configurations/metastages on the 
cd (declared in a DB, text file, whatever) and just have portage 
assemble the actual tarball on the fly by deriving the packages from 
the current live environment or using its knowledge to create the 
missing ones. That saves us at least a few gigs on the mirrors. Once 
portage gets a faster DB backend, the limiting factor will become the 
expressiveness (conflicting packages, glep 23, glep 21, glep 19, glep 
22 (especially) , emerge -e world && auto-use ...) and the speed drop 
that comes with increasing that, (we go from a tree walking algorithm 
(nothing fancy) to actual reasoning/planning, which is a bit more 
challenging :-)

Best regards,

Pieter Van den Abeele


--
gentoo-dev@g.o mailing list