Re: [gentoo-dev] Re: where goes Gentoo?

[Chris Gianelloni]

On Thu, 2005-08-04 at 14:37 -0500, Brian D. Harring wrote:
> Elaborate on what you explicitly want out of portage please- the
> domain concept (aside from being useful design wise) *should* allow
> groupping of boxes (groupping of domains really) behind it, so you can
> effectively have a set of boxes, pushing changes to each.
>
> Mind you no code written, but current design is intended to allow
> remote chunks to be swapped in/out of portagelib on the fly
> (including the actual portage configuration).

The only things I could see being needed out of portage itself is the
ability to control "emerge" commands remotely, such as forcing an update
of apache to $version to resolve a vulnerability.

Besides the back-end portage pieces, there would need to be a front-end
interface for performing these tasks.

> Better angle of discussion rather then "we aren't there yet" is the
> specifics of what is needed to *get* there in peoples opinion.

Agreed completely.

Some things I could see as needed:

1. applying updates on any file that is under CONFIG_PROTECT where the
md5/file-size matches that in /var/db for the file without user
interaction
2. automatic removal of files under CONFIG_PROTECT where the
md5/file-size matches that in /var/db during unmerge

> It's not an overnight thing, glep19 (stable portage tree) addresses a
> chunk of concerns when/if it's implemented, but I'm a bit more
> interested in the the other tools people desire alongside.

As am I.  The Installer is one such project.  We do not have any project
that I am aware of that is designed to resolve the problem of remotely
managing a server.  There is nothing for pushing config changes/package
updates/new packages.  There would need to be some interface for doing
these things.  Stop by any trade show, such as LWE, and you'll see guys
pushing their wares on remotely managing Linux.  We should provide
something like this ourselves.

eg. If I want to change the subnet mask or default router on 50 machines
on my network, I should be able to do so via a simple interface and have
the work done automatically.

> Re: a drop-in solution, considering that gentoo is effectively all
> over the map (seriously, look at the tree), define the profile for the
> drop-in; drop-in ftp, drop-in web server, drop-in mosix node... etc.

I meant a drop-in management solution, not a specific set of server
profiles, though those could be created with the Installer.  In fact, I
see the Installer as one of the first pieces of the framework necessary
for deployment and management of a large number of servers.  Once the
netfe interface is completed with the Installer, you will be able to PXE
boot your server and have it load a specific installer profile, and it
will install Gentoo to those specifications.  Beyond that, we lose
control of the server and must manually perform all other actions.

> Specifics...
>
> Hell, I have yet to see what I would define as a proper solution for
> config manamagent for N gentoo boxes.  NFS solution possibly, but that
> seems a bit hackish to me.

There isn't a proper solution yet.  Honestly, something like a
repository holding configuration information with revision control would
probably be best, so you can revert changes.  There are quite a few
systems like this out for Red Hat and others, but nothing that is
Gentoo-specific, or even Gentoo-capable, as far as I know.  We should
beat people to the punch and design one ourselves.

The main things we need to provide are:

Provisioning - building a server from bare metal to some pre-determined
state
Management - being able to make changes to existing servers without
manually logging into each to make the changes
Updates - this somewhat goes with management, but a facility for
disseminating patches or updated packages to servers

> Moot point frankly, considering we're all volunteers; someone
> *could* get off their butts and start up an attempt to provide hand
> holding (effectively what you're coloring the management arg as)
> services, but even if they did, the followup arg would be that you
> can't yet trust this new support company, because they're new.
> Etc.

Not entirely moot, as a company could be formed in cooperation with the
Foundation, as I stated earlier in the thread.  This symbiotic
relationship would give the new company a bit more credit, as it will be
supported by the Foundation.  This could be a Foundation-owned company,
or a completely separate entity.  Anyway, this isn't so much my point,
as many people *are* willing to forgo having a human voice on the end of
a phone.

> Basically, we don't have control over that portion, so... what
> can be mangled that we *do* have control over, and has an effect?

Our tools.  Currently, we have very few "Gentoo tools" used for managing
a system.  We would need to define the requirements for these tools, and
then work on ways of getting them built.  It's like I said, I think the
primary weakness in Gentoo's enterprise adoption is the need for each
company to reinvent the wheel on their own deployment.  If we had a set
of extensible tools for managing Gentoo machines, then companies would
have a framework for building upon to meet their own needs.  Why does
everyone, for example, need to invent their own way of adding users to
their network?  Why can't we provide some method and allow them to
customize it and extend it?

> Mentioned management tools, well, get into specifics; pxe network
> installs/imaging?  Single tree/cache for N servers?  Ability to push
> updates out to a specific box, or set of servers?  Integration of
> portage contents db with IDS tools?

PXE installs is on its way.  Being able to share the tree/caches would
definitely be of benefit.  I already discussed updates.  I hadn't even
considered the IDS integration, but that is an awesome idea.  How about
configuration file management?  Asset management?  Inventory database?
How about a "remote assistance" feature?  Since Gentoo is not only used
on servers, but could also be deployed on the workstation, we should
also provide tools for managing and supporting them, too.  What about
some form of policy enforcement?  Things like turning on Remote Desktop
Sharing in KDE/Gnome, so IT staff can assist users with issues.

> > Portage really has nothing to do with deployment or management.  In
> > fact, the only thing it really does is package management, which is
> > probably why it is called a package management tool, and not an
> > enterprise resource manager.
>
> Any enterprise resource manager is going to have to fool with pkgs at
> some point- that's my line of interest in this.

Correct.  I think my meaning was that we need to look at things
*besides* package management.  You guys seem to already have a good idea
of the things we need and I've seen progress towards making portage more
enterprise-friendly with some of the features planned for the future.

The main thing we need is a powerful portage API that allows complete
control of portage without using "emerge" at the command line.

> > Gentoo is currently unmaintainable at this scale without a significant
> > investment in infrastructure and development to make the system
> > manageable.  Think of it this way, if I can pay 4 developers to work on
> > this project for 6 months, and each developer makes $50,000 a year, or I
> > can pay Novell $100,000 and have the system in place in 2 weeks, which
> > do you think I would do?  This is the exact reason why Gentoo is not
> > used in the enterprise more.  There is simply too high a barrier of
> > entry into making a usable and manageable Gentoo deployment.

> Or, you find a collection of trained coder monkeys who are oddballs
> who might have an interest in implementing this stuff on their own
> time, and try to nudge them in the correct direction; no, this isn't a
> solution, but again, having an ent. solution (going by your statement)
> isn't going to be funded by anyone.

I meant this to mean $company pays developers to implement this for
themselves, whereas Novell/Red Hat have already paid for most of this
work on their own distributions.  The idea being that we are much more
likely to get enterprise adoption if we have some tools in place, even
if rudimentary in comparison, where currently we have nothing.

> Ok, fine.  So it goes.
>
> Meanwhile, reiterating my point, I'd rather see a discussion of what
> people *want* in the way of tools, then "we aren't there yet".
> Generally known that you have to roll your own somewhat for tools,
> well, would rather know what people want then see then another round
> of kicking the dead horse.

Quite simply:

Some form of GUI (and console) tools capable of controlling every aspect
of any given set of Gentoo servers within an enterprise, from birth
until death.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux