Re: [gentoo-dev] Re: where goes Gentoo?

[Brian D. Harring]

On Thu, Aug 04, 2005 at 05:31:43PM -0400, Chris Gianelloni wrote:
> The only things I could see being needed out of portage itself is the
> ability to control "emerge" commands remotely, such as forcing an update
> of apache to $version to resolve a vulnerability.

The requirements of portage, or whatever component supplies 
(essentially) pkg management of remote boxes is going to be a bit more 
complex then just pushing emerge commands out; aside from config data, 
it'll probably centralize the vdb type contents somewhere, let alone 
avoiding N copies of the ebuild tree on each server.

Basically, whatever daemon is running clientside for all of this will 
have to support a good bit of handing off commands to portage, hence 
the interest (since from my point of view, it's a starting point).


> Some things I could see as needed:
>
> 1. applying updates on any file that is under CONFIG_PROTECT where the
> md5/file-size matches that in /var/db for the file without user
> interactio

That would have to be determined prior to starting the update push.
I'd think basically a CONFIG_PROTECT limited scan of boxes to be 
updated, verifying things are in order according to the vdb (whether 
remote or local to that box) probably would fly.


> 2. automatic removal of files under CONFIG_PROTECT where the
> md5/file-size matches that in /var/db during unmerge

current vdb implementation relies on md5/file-size, future should rely 
on refcount, and be a good bit more configurable.


> > It's not an overnight thing, glep19 (stable portage tree) addresses a
> > chunk of concerns when/if it's implemented, but I'm a bit more
> > interested in the the other tools people desire alongside.
Offhand, responding to my own snippet, I'd love to know what's going 
on with glep19...

>
> As am I.  The Installer is one such project.  We do not have any project
> that I am aware of that is designed to resolve the problem of remotely
> managing a server.  There is nothing for pushing config changes/package
> updates/new packages.  There would need to be some interface for doing
> these things.  Stop by any trade show, such as LWE, and you'll see guys
> pushing their wares on remotely managing Linux.  We should provide
> something like this ourselves.
>
> eg. If I want to change the subnet mask or default router on 50 machines
> on my network, I should be able to do so via a simple interface and have
> the work done automatically.

Approach I've been thinking about (that fits semi-neatly exempting 
collision-protect) is essentially config pkgs, binding them on the fly 
to pkgs being pushed out.  Essentially, base apache pkg (that out of 
an ebuild tree), with it's depend tweaked automatically to pull in a 
matching configuration pkg.

Pushing out config updates wouldn't be too hard if handled in this 
manner, although generation of the config pkgs themselves would be a 
bit tricky.


> > Re: a drop-in solution, considering that gentoo is effectively all
> > over the map (seriously, look at the tree), define the profile for the
> > drop-in; drop-in ftp, drop-in web server, drop-in mosix node... etc.
>
> I meant a drop-in management solution, not a specific set of server
> profiles, though those could be created with the Installer.  In fact, I
> see the Installer as one of the first pieces of the framework necessary
> for deployment and management of a large number of servers.  Once the
> netfe interface is completed with the Installer, you will be able to PXE
> boot your server and have it load a specific installer profile, and it
> will install Gentoo to those specifications.  Beyond that, we lose
> control of the server and must manually perform all other actions.
Niete.

> > Specifics...
> >
> > Hell, I have yet to see what I would define as a proper solution for
> > config manamagent for N gentoo boxes.  NFS solution possibly, but that
> > seems a bit hackish to me.
>
> There isn't a proper solution yet.  Honestly, something like a
> repository holding configuration information with revision control would
> probably be best, so you can revert changes.  There are quite a few
> systems like this out for Red Hat and others, but nothing that is
> Gentoo-specific, or even Gentoo-capable, as far as I know.  We should
> beat people to the punch and design one ourselves.
>
> The main things we need to provide are:
>
> Provisioning - building a server from bare metal to some pre-determined
> state
Installer...

> Management - being able to make changes to existing servers without
> manually logging into each to make the changes
Domain class should provide for it
> Updates - this somewhat goes with management, but a facility for
> disseminating patches or updated packages to servers
Same as above


> Our tools.  Currently, we have very few "Gentoo tools" used for managing
> a system.  We would need to define the requirements for these tools, and
> then work on ways of getting them built.  It's like I said, I think the
> primary weakness in Gentoo's enterprise adoption is the need for each
> company to reinvent the wheel on their own deployment.  If we had a set
> of extensible tools for managing Gentoo machines, then companies would
> have a framework for building upon to meet their own needs.  Why does
> everyone, for example, need to invent their own way of adding users to
> their network?  Why can't we provide some method and allow them to
> customize it and extend it?
glep27 comes to mind re: users, although that's not management of 
samba users (fex).


> > Mentioned management tools, well, get into specifics; pxe network
> > installs/imaging?  Single tree/cache for N servers?  Ability to push
> > updates out to a specific box, or set of servers?  Integration of
> > portage contents db with IDS tools?
>
> PXE installs is on its way.  Being able to share the tree/caches would
> definitely be of benefit.  I already discussed updates.  I hadn't even
> considered the IDS integration, but that is an awesome idea.  How about
> configuration file management?

Configuration file management, as long as it's centralized, can be 
slightly bastardized as pkgs for pushing/updating.  If that's the 
case, it should be possible to avoid reinventing the wheel for 
handling it- hence the IDS comment.  Verification of config's prior to 
stomping them on an upgrade.

>  Asset management? Inventory database?
No good answer on that one, since it's outside the ken of what my area 
of interest (portage) :)
Offhand, I'd expect whatever method is used to push commands down via 
the domain class, probably can be extended to add these additional 
knobs.  It really depends on what you're trying to query though, 
cpuinfo/df, or license management...


> > > Portage really has nothing to do with deployment or management.  In
> > > fact, the only thing it really does is package management, which is
> > > probably why it is called a package management tool, and not an
> > > enterprise resource manager.
> >
> > Any enterprise resource manager is going to have to fool with pkgs at
> > some point- that's my line of interest in this.
>
> Correct.  I think my meaning was that we need to look at things
> *besides* package management.  You guys seem to already have a good idea
> of the things we need and I've seen progress towards making portage more
> enterprise-friendly with some of the features planned for the future.
>
> The main thing we need is a powerful portage API that allows complete
> control of portage without using "emerge" at the command line.

Heh, what, current api isn't usable? :)
Yeah, api is an area needing improvement.


> Some form of GUI (and console) tools capable of controlling every aspect
> of any given set of Gentoo servers within an enterprise, from birth
> until death.
Oh... just that.  'k. :)

re: the remote assist/control of a box, I'd wonder what could be 
handled via ldap (auth) and use flag...
~harring