Re: [gentoo-dev] Enterprise Installer [was "LWE Aftermath" on -core]

[Chris Gianelloni]

On Fri, 2004-08-06 at 15:26, Joshua J. Berry wrote:
> No, a profile a la Gentoo's Portage profile probably isn't ... I just used the
> term "profile" to mean a specific set of installed packages and configuration
> options (e.g. have a "mailserver" profile, a "webserver" profile and perhaps a
> "workstation" profile).

You mean something more of an installation type, rather than a "profile"
then... Make sure we don't re-use terminology that could be ambiguous in
meaning.

> You could certainly use Portage profiles for that purpose though, and I think it
> might be a good place to start, at least until requirements dictate we find
> something better.

Not really... unless you wanted everything in your "profile" to be
considered the "system".

> > The pre-compiled
> > binaries are a must.  I could see this working in many ways, all of
> > which would need to be discussed.  A proper "rollout" tool would not
> > require a CD, so there would be no need to involve catalyst for such a
> > task, though there's no reason to not allow it.  I would instead see
> > catalyst taking the role of creating the binaries.
>
> I guess I don't quite understand what Catalyst's purpose is then.  Is it not to
> create boot CDs for installation?

Catalyst can be used for many things.  It makes GRP sets, LiveCD's, and
a tinderbox.  A LiveCD does not have to be an installation CD.  In fact,
I am working now to extend catalyst to create a GameCD.

I guess I am saying the process to create a LiveCD is a bit more
involved than needed for most cases.  Using a network boot method plus
some scripts would be much quicker and cleaner.  It also wouldn't
require lugging around a CD, which means the CD must be kept up-to-date
with changes and other such problems.  Definitely not the best way for
the enterprise to go about it.

> > Installing new
> > machines would be carried out using industry standard methods, such as
> > PXE to boot an image that downloads from TFTP and loads up a NFS root
> > with an installer image.
>
> I like this idea better than a boot CD, but never having tried it myself, I
> don't know how it would be setup or how practical it would be for users that are
> just interested in setting up, say, a few machines at their small business in
> this way.

This isn't designed for the guy wanting a few machines, though he would
be able to utilize it.  It would probably still be quicker than creating
a custom boot CD.  Especially since if we were to create it, we would
make it simple for the user.  It would probably end up no harder than
creating a catalyst .spec file to begin with, so why take the time to
make a CD out of it?

Also, stick with scope here.  Are we talking the guy with 10 machines or
are we talking the ENTERPRISE where we're considering hundreds or
thousands of machines.  Having to go to each of a thousand machines with
a CD can be tedious.

> Some admins may wish to perform their own QA on the Portage tree before pushing
> things out to their userbase, or perhaps create custom ebuilds for proprietary
> software they have purchased and want to run on Gentoo.

What makes this different than one of several methods:

- The admin runs his own rsync mirror with only his "QA certified"
ebuilds in it.
- The admin NFS mounts his "QA certified" portage tree.
- The admin pushes out his "QA certified" portage tree as an overlay.

Personally, I think the first is the best.  Remember, the admin does not
have to offer up *our* tree for rsync.  If he decides to standardize on
KDE and doesn't want anyone running games, he can rm -rf
/usr/portage/gnome-* and /usr/portage/games-* and even add them to his
RSYNC_EXCLUDES.


> Some people have also requested the ability to deliver a stripped-down version
> of the tree (e.g. removing server-ish ebuilds from trees that will only be used
> with workstations), so that it's harder for those of their end users who have
> root to do something silly like install a DHCP server on a workstation.

See above.  We have not denied this ability since the introduction of
the SYNC variable.

> By "stable", are you talking about the snapshots a la Gentoo Enterprise?  (i.e.
> Take a stable snapshot every N months and release it.)  I think we would
> probably be OK without this, at least for those admins who choose to do their
> own QA.  After all, staying fairly consistently up-to-date is another advantage
> to Gentoo which some may wish to take advantage of.

Installing a Gentoo "release" should be the same no matter when you
install it.  The versions should be the same.  The admin should know
what to expect from our distribution.  If you follow my thread (way back
a couple weeks ago) about having a set of -release trees and a -current
tree, you'll see what I mean.  At no point are we limiting the user's
choice by offering a frozen -release tree for each release.  In fact, we
are giving the user more options.

> I have no money, but I do have time (at least during the summer), and I have a
> certain amount of personal interest.  (I'd like my school to start using Gentoo,
> and I think having a tool like this might convince them.)

I'm not sure you understand the scale of such a project.  Something like
this would require an enormous dedication and a very far-reaching plan. 
To be honest, even if you're just starting school, I wouldn't expect to
see this even at a remotely close to completed stage by the time you
graduate without some serious financial and manpower backing.

> If enough people are willing to work on this, we should start (yet) a(nother)
> project.

Instead, what needs to be done rather than start yet another project
which will go dormant in 6 months, is to get everyone interested
involved, including any other projects that would be encompassed, such
as the installer guys, and actually put something together and stick
with it.  The real problem is that these sort of things are pretty
boring to bring about.  Interest wanes quite easily when working on a
project of such a scope.  Rather than come up with these huge, elaborate
plans, what needs to be done is baby steps need to be achieved.  Take
everything step by step.

I still think our first step should be the creation of a -release tree
and that's it.  Forget claiming support on it or any of that nonsense,
since we can't do it and we know it.  We won't even claim to back port
any fixes, because we probably won't.  We can add fixes that *are* back
ported, but we won't guarantee any availability on them.  What this
does, is it creates a definitive itch for people to scratch.

> This means--and this has been brought up repeatedly before--there needs to be
> some sort of for-profit company, separate from the non-profit, that has at least
> a few full-time Gentoo devs on its staff.  Or so it seems to me.

Gentoo Linux is run by a not-for-profit company for a reason.  We do not
want corporate interests guiding us.

What would need to be done, is a separate entity would need to be
established.  This entity could be owned by the Gentoo Foundation, or
completely separate.  It could be manned by Gentoo devs, or just users,
it doesn't matter.  The main problem with doing this and PAYING people
to work on the system is the sheer amount of capital that would be
required to operate such a venture.  I would guess that you would be
paying the entire staff for at least 2 years just to get to the point of
being able to offer a supportable product.  After all, every single bit
of the supported tree would require developers to back port fixes.

More than likely, only a certain subset of the tree would be supported,
as attempting to support the entire thing would be a daunting task. 
Perhaps a group of people simply willing to support all of the packages
offered via GRP, including back porting fixes, would be a good start.

> There may even be (and probably are) a set of people out there offering Gentoo
> to their clients as part of their business.  Perhaps we should start keeping a
> list of these people.

I think we should.  I know there are people using Gentoo for this sort
of thing.

I still like the idea that was proposed of some form of support coop.

> > I believe that to achieve these goals, we have to start by taking baby
> > steps.  Perhaps we should bring up some things at the next manager's
> > meeting?  To be honest, it looks like the first steps are going to be
> > the easiest, and at the same time the hardest.  We have to decide where
> > we want to go and lay out a plan.  The hardest part is we have to stick
> > to that plan and not falter.
>
> The first step is always to figure out where you want to go ... I think we're
> doing that now.  Then we just need to break the problem down into small pieces.

We figured out where we wanted to go the last time this came up... and
the time before that... and the time before that... and the....

I've given a good starting point, a frozen tree.  Since such a thing
would require buy-in from -releng (not a problem), -infra and the mirror
admins, and the management, it needs to be brought up at a manager's
meeting and voted on, preferably with some idea of the additional mirror
space and some initial feedback from all the parties involved.

> If anyone is interested, I can perhaps draft up some thoughts/the beginnings of
> a raodmap for getting this done.
>
> > > So ... what are your thoughts on this?
> >
> > We need to move on it, and not talk about it, then let the topic die off
> > and forget about it as has been going on for months now.
>
> OK.  Let's go. :)
>
> >
> > > On a related note, avenj did mention the Installer project to me, and gave me a
> > > link to their CVS, but it hasn't been touched in a few months.  Anyone know what
> > > their status is, and what specifically they're trying to accomplish?
> >
> > I believe their charter was to create a nice installer that could be
> > used to install Gentoo in various fashion, using GRP as a base and
> > allowing for rapid deployment of servers and workstations based on
> > pre-defined criteria.
>
> That sounds like it's the first (big) step then.  Let's get an installer
> working.

What would the installer *do* exactly?  That is a VERY large step to
take... or at least I think so.  Does the installer perform a stage1
install? stage3? GRP? GRP + more packages?  Does it use the portage
snapshot each arch lead makes for his release?

Wouldn't it be easier to do with a frozen -release tree? (In case you're
not getting it, I want a frozen tree... *grin*)

-- 
Chris Gianelloni
Release Engineering QA Manager/Games Developer
Gentoo Linux

Is your power animal a penguin?