Re: [gentoo-dev] Enterprise Installer [was "LWE Aftermath" on -core]

[Chris Gianelloni]

On Fri, 2004-08-06 at 17:34, Joshua J. Berry wrote:
> No, I do mean a profile-ish thing ... install is just the first step in maintenance.
> Suppose you have a bunch of preinstalled workstations and you want to push out
> another app.  How do you do it?  Add it to the profile-ish thing.

So you mean something like Red Hat's channel subscriptions?  Honestly,
they have a good idea.  Have you ever played with their Satellite Server
product?  It is essentially a local copy of RHN which also allows you to
create custom channels to subscribe systems to and also allows you to
easily manage your kickstart configurations for building new systems.

> > This isn't designed for the guy wanting a few machines, though he would
> > be able to utilize it.  It would probably still be quicker than creating
> > a custom boot CD.  Especially since if we were to create it, we would
> > make it simple for the user.  It would probably end up no harder than
> > creating a catalyst .spec file to begin with, so why take the time to
> > make a CD out of it?
>
> The point I was trying to make is, we should--eventually--support both options.
> We don't need to do it immediately, of course, but it should be there.

I don't think I am following you.  What is "the little guy" option?  At
what point does it become easier to not use DHCP and PXE?  Is it even
worth the effort to "deploy" on that number of machines, or are you
speaking more of a simple installer?  The installer is within the scope
of the enterprise product.  Nothing is stopping someone from using it,
even on a single machine.

> > Also, stick with scope here.  Are we talking the guy with 10 machines or
> > are we talking the ENTERPRISE where we're considering hundreds or
> > thousands of machines.  Having to go to each of a thousand machines with
> > a CD can be tedious.
>
> Enterprise, for now, which can eventually be scaled down to work with the guy
> who has 10 machines and want to use it at home.

There's nothing stopping the person with 10 machines from using the
product as intended.  I don't think there's any need to "scale down" any
properly designed product.  Instead, it should be usable on anything
from 2 machines (one server and one client) on up, to only a single
machine if only using the installer and none of the "kickstart" type of
deployment tools.

> The first is pretty much what I was suggesting.
>
> I realize this can be done now, but there should--eventually--be tools to make
> management of custom trees easier.

I'm not sure how much easier it can be that copying an ebuild, but I'm
with you.  If you see a need and it is something that should be managed,
then by all means lets do it.

> > I'm not sure you understand the scale of such a project.
>
> I think I do.  I explored this idea (on a more limited scope, of course) when I
> was working for Cal Poly's CSC dept last year.

Well, understand that Gentoo is very very big and very very diverse.  We
extend from Argentina to Alaska and everywhere in between.  Our users
speak every imaginable language and have any number of wants and
desires.  We also provide an enormous amount of software via portage,
which needs to be managed.

> > Something like this would require an enormous dedication and a very
> > far-reaching plan.  To be honest, even if you're just starting school, I
> > wouldn't expect to see this even at a remotely close to completed stage by the
> > time you graduate without some serious financial and manpower backing.
>
> I understand that.  But if we don't start somewhere, such a project will never
> be completed.

I agree.  I also think that it is a long and painful road and am simply
not rushing into anything with a fool's hope that it will be easy.

> > > If enough people are willing to work on this, we should start (yet) a(nother)
> > > project.
> >
> > Instead, what needs to be done rather than start yet another project
> > which will go dormant in 6 months, is to get everyone interested
> > involved, including any other projects that would be encompassed, such
> > as the installer guys, and actually put something together and stick
> > with it.
>
> How is this different from creating a project?

Creating a project does nothing but give something a name.  We already
have a Gentoo Server project and a Gentoo Enterprise project, not to
mention at least one GLEP which all fall under the scope of creating a
corporate-friendly Gentoo.

> > The real problem is that these sort of things are pretty boring to bring
> > about.  Interest wanes quite easily when working on a project of such a scope.
> > Rather than come up with these huge, elaborate plans, what needs to be done is
> > baby steps need to be achieved.  Take everything step by step.
>
> Portage wasn't written in a day.  I understand your point about baby steps, but
> how do you expect to get anywhere if you have no idea of where you're going?

I'm still fairly new as a developer, but I have seen tons of great ideas
brought up with much fever and passion behind it, only to watch it
fizzle out and die a few weeks later once it runs out of everyone's
minds.  Having grand plans is great if you're planning on exploring
Egypt.  For a software project, the best course of action is to decide
where you want to go and implement a plan.

I definitely don't have the answers here.  I just see things that work
and like to point them out... *grin*

> > The main problem with doing this and PAYING people
> > to work on the system is the sheer amount of capital that would be
> > required to operate such a venture.  I would guess that you would be
> > paying the entire staff for at least 2 years just to get to the point of
> > being able to offer a supportable product.  After all, every single bit
> > of the supported tree would require developers to back port fixes.
>
> I don't know enough about how companies work, or how much it would take to get
> the Portage tree into a "supportable" state.  If this is the case, then perhaps
> creating a separate for-profit company wouldn't be feasible, and the list of 3rd
> party supporters is a better idea.

It could be feasible, provided there was enough financial interest to
understand that they would be losing their a** for the first couple
years, seeing little to no returns on their investment.  The portage
tree isn't in a sad shape at all, but we rely on our ability to move to
newer versions and technologies quickly to help us overcome problems
such as security, rather than spending lots of developer time fixing
things from upstream.  I am not trying to belittle anyone's efforts,
because everyone works so hard.  What I mean is that sometimes (most
times?) it is easier to upgrade from foo-1.0 to foo-1.1 than to fix
foo-1.0's flaws.  This doesn't jive well with corporate interests, which
generally want as little change as possible.

> > We figured out where we wanted to go the last time this came up... and
> > the time before that... and the time before that... and the....
>
> Where's the plan, then?  Has anybody written up a coherent plan, broken the
> tasks down into small steps, and posted it somewhere?

Nobody ever got that far.  Instead there was lots of "great ideas" and
"let's do that" and then as soon as the thread died on -dev (or -core)
it was out of everyone's minds.

> I certainly don't have a problem with a frozen tree; I think it's a good idea.

Great!  Should we bring it up at an upcoming manager's meeting?

> But I guess I'm not entirely sure how a frozen tree would help us here ... I
> think we might be talking about two different things.  I was talking mainly
> about providing a way for large sites to painlessly configure/install/maintain
> Gentoo on lots of machines.

How do you install/maintain/configure a large number of machines when
your base is ever changing?  Install a machine using the 2004.2 CD right
now.  Wait 2 hours, then install another machine.  Are the package
versions the same?  Why not?  If the versions are the same, then 5
months from now someone can install from that CD and get the same
results.  Reproducible results are a necessity for companies planning
for their infrastructure.  It also makes things a lot easier to support
when you know what to expect.

> > What would the installer *do* exactly?  That is a VERY large step to
> > take... or at least I think so.  Does the installer perform a stage1
> > install? stage3? GRP? GRP + more packages?  Does it use the portage
> > snapshot each arch lead makes for his release?
>
> Well, it is a big step.  We're not really ready to start doing that yet.
>
> Like you said, we need to take baby steps.  Let's get/make a list of those for
> the installer.

Honestly, I don't see the installer as a first step, but that is totally
up for discussion.  I tend to think that management tools would be a
better place to focus, along with working to implement ideas that have
already been proposed.  We should be able to have a complete binary
install of Gentoo done using GRP in about 20-30 minutes, but it takes
almost an hour due to the number of manual steps required, and compiling
a kernel.  There is a GLEP for the introduction of compiled kernels. 
Would that not be beneficial for an installer and an enterprise Gentoo?

> > Wouldn't it be easier to do with a frozen -release tree? (In case you're
> > not getting it, I want a frozen tree... *grin*)
>
> OK, so let's make a frozen tree.  But that doesn't have much to do with
> configuration/installation/maintenance of lots of machines. ;)

It doesn't really.  It just lays a strong framework to build upon.

I don't have intentions for even starting a frozen tree by the next
release, but I would like to shoot for 2005.0, if we can get buy-in from
management and developers.

I definitely would love to see more work being done in making Gentoo
easier to install and use in an enterprise.

-- 
Chris Gianelloni
Release Engineering QA Manager/Games Developer
Gentoo Linux

Is your power animal a penguin?