Re: [gentoo-dev] Enterprise Installer [was "LWE Aftermath" on -core]

[Joshua J. Berry]

On Fri, Aug 06, 2004 at 08:08:32PM -0400, Chris Gianelloni wrote:
> On Fri, 2004-08-06 at 17:34, Joshua J. Berry wrote:
> > No, I do mean a profile-ish thing ... install is just the first step in maintenance.
> > Suppose you have a bunch of preinstalled workstations and you want to push out
> > another app.  How do you do it?  Add it to the profile-ish thing.
>
> So you mean something like Red Hat's channel subscriptions?  Honestly,
> they have a good idea.  Have you ever played with their Satellite Server
> product?  It is essentially a local copy of RHN which also allows you to
> create custom channels to subscribe systems to and also allows you to
> easily manage your kickstart configurations for building new systems.

I haven't ... I stopped using Red Hat myself sometime around RH8, and I never
did use their RHN system.

> > The point I was trying to make is, we should--eventually--support both options.
> > We don't need to do it immediately, of course, but it should be there.
>
> I don't think I am following you.  What is "the little guy" option?  At
> what point does it become easier to not use DHCP and PXE?  Is it even
> worth the effort to "deploy" on that number of machines, or are you
> speaking more of a simple installer?  The installer is within the scope
> of the enterprise product.  Nothing is stopping someone from using it,
> even on a single machine.

mmm.  You're probably right.  It's been a while since I looked at PXE myself, so
I may be remembering it as harder/more daunting than it really is.

> > The first is pretty much what I was suggesting.
> >
> > I realize this can be done now, but there should--eventually--be tools to make
> > management of custom trees easier.
>
> I'm not sure how much easier it can be that copying an ebuild, but I'm
> with you.  If you see a need and it is something that should be managed,
> then by all means lets do it.

Except it's not just copying the ebuild, it's copying the ebuild, managing the
digests, copying the patches ... ;)

> > > Something like this would require an enormous dedication and a very
> > > far-reaching plan.  To be honest, even if you're just starting school, I
> > > wouldn't expect to see this even at a remotely close to completed stage by the
> > > time you graduate without some serious financial and manpower backing.
> >
> > I understand that.  But if we don't start somewhere, such a project will never
> > be completed.
>
> I agree.  I also think that it is a long and painful road and am simply
> not rushing into anything with a fool's hope that it will be easy.

Oh no, I never said it would be easy.  Just that we should do it. ;)

> > > The real problem is that these sort of things are pretty boring to bring
> > > about.  Interest wanes quite easily when working on a project of such a scope.
> > > Rather than come up with these huge, elaborate plans, what needs to be done is
> > > baby steps need to be achieved.  Take everything step by step.
> >
> > Portage wasn't written in a day.  I understand your point about baby steps, but
> > how do you expect to get anywhere if you have no idea of where you're going?
>
> I'm still fairly new as a developer, but I have seen tons of great ideas
> brought up with much fever and passion behind it, only to watch it
> fizzle out and die a few weeks later once it runs out of everyone's
> minds.  Having grand plans is great if you're planning on exploring
> Egypt.  For a software project, the best course of action is to decide
> where you want to go and implement a plan.

I've seen that happen quite a bit as well.  I don't really have any solution
other than find some way to keep it interesting so people will work on it.

I myself am guilty of this (there haven't been too many GLSAs with my name on
them recently :( ).  If I find a solution to this I'll let everyone know. ;P

> > I don't know enough about how companies work, or how much it would take to get
> > the Portage tree into a "supportable" state.  If this is the case, then perhaps
> > creating a separate for-profit company wouldn't be feasible, and the list of 3rd
> > party supporters is a better idea.
>
> It could be feasible, provided there was enough financial interest to
> understand that they would be losing their a** for the first couple
> years, seeing little to no returns on their investment.  The portage
> tree isn't in a sad shape at all, but we rely on our ability to move to
> newer versions and technologies quickly to help us overcome problems
> such as security, rather than spending lots of developer time fixing
> things from upstream.  I am not trying to belittle anyone's efforts,
> because everyone works so hard.  What I mean is that sometimes (most
> times?) it is easier to upgrade from foo-1.0 to foo-1.1 than to fix
> foo-1.0's flaws.  This doesn't jive well with corporate interests, which
> generally want as little change as possible.

My experience, at least with security bugs, has been that upstream developers in
general tend to keep maintenance releases and new feature releases separate.
Oftentimes there will be an upgrade to a package that contains only the security
fixes, and no other new code.  Of course, this isn't true for all packages, but
it is true for most of the ones I've worked on.

Sometimes the security team will backport fixes (e.g. the rsync vulnerability
that came out not too long ago) because we don't want to push a later version
out for stability reasons.  But again, that's just security, I don't know how
others handle it.

> > > We figured out where we wanted to go the last time this came up... and
> > > the time before that... and the time before that... and the....
> >
> > Where's the plan, then?  Has anybody written up a coherent plan, broken the
> > tasks down into small steps, and posted it somewhere?
>
> Nobody ever got that far.  Instead there was lots of "great ideas" and
> "let's do that" and then as soon as the thread died on -dev (or -core)
> it was out of everyone's minds.

There's the first problem.  Nobody's even really explored the feasibility of
doing something like this ... what exactly--and by exactly, I mean right down to
the baby steps--would need to be done?

> > I certainly don't have a problem with a frozen tree; I think it's a good idea.
>
> Great!  Should we bring it up at an upcoming manager's meeting?

Go for it.  I'm not a manager but FWIW you have my moral support. :)

> > But I guess I'm not entirely sure how a frozen tree would help us here ... I
> > think we might be talking about two different things.  I was talking mainly
> > about providing a way for large sites to painlessly configure/install/maintain
> > Gentoo on lots of machines.
>
> How do you install/maintain/configure a large number of machines when
> your base is ever changing?

You use your own internal-QA-approved Portage tree.

> Install a machine using the 2004.2 CD right
> now.  Wait 2 hours, then install another machine.  Are the package
> versions the same?  Why not?  If the versions are the same, then 5
> months from now someone can install from that CD and get the same
> results.  Reproducible results are a necessity for companies planning
> for their infrastructure.  It also makes things a lot easier to support
> when you know what to expect.

I agree, which is why I think frozen trees, or site-specific Portage trees are a
good idea.

> Honestly, I don't see the installer as a first step, but that is totally
> up for discussion.  I tend to think that management tools would be a
> better place to focus, along with working to implement ideas that have
> already been proposed.

Well, the installer is a management tool of sorts.

But really, the first step is finding out what the first step is. ;)

> We should be able to have a complete binary
> install of Gentoo done using GRP in about 20-30 minutes, but it takes
> almost an hour due to the number of manual steps required, and compiling
> a kernel.  There is a GLEP for the introduction of compiled kernels.
> Would that not be beneficial for an installer and an enterprise Gentoo?

Yes, it would, in the same way that frozen trees are good.  I don't know how
feasible this would be, though ... how do you handle all the different hardware
configurations?  We could very easily end up with bloated kernels a la the
binary distributions.

I think I would prefer that each site compile one kernel for each set of
machines it needs, and the compiled kernels get installed separately.  I have no
idea how this would work, however. :)

> I definitely would love to see more work being done in making Gentoo
> easier to install and use in an enterprise.

So let's at least start a plan, and see where we can go from there.

-----------------------------------------
Joshua J. Berry

"I haven't lost my mind -- it's backed up on tape somewhere."
    -- /usr/games/fortune

NOTE: Please do not submit this email address to any mailing
lists or websites without prior permission.  Thank you.