[gentoo-server] Community Development of Gentoo Server Mgmt Tools for VMware Clusters/etc - Puppet modules, github collaboration, etc - Fork of Dev Topic: Avoiding Urgent Stabilizations

[Matthew Marlowe]

Reader note -- Direct all respones to gentoo-server ML to avoid flooding 
gentoo-dev.

For those on server-ml who missed dev-ml:

I'm starting to put together a portage/stable server configuration for a large 
number of gentoo VM's that will eventually be hosted on a VMware ESX 4.1U1 
cluster - with the goal of limiting major changes to once/year and otherwise 
only applying security/minimum necessary updates.  I doubt it will be easy but 
I'm doing my best at it :)

As part of that I'm maintaining on github several related repositories, 
including portage mask/use/config files, cluster management utilities, etc.

     https://github.com/deploylinux

I've also started to document work on my blog:

     http://www.deploylinux.net/matt

I'm not currently planning to utilize a separate overlay for packages/ebuilds, 
but it's not out of the question.

I'd be happy to work with any other devs or users w/ similiar interests or 
production networks to manage.....github makes group development relatively 
easy.

On Friday, February 25, 2011 03:37:36 am Ed W wrote:
> I maintain a, likely much smaller, number of VMs using linux vservers.
> The approach here is to almost cut each machine down to a chroot that
> runs only one (or thereabouts) interesting service.  To do this I have
> found customised portage profiles to be the under-plugged secret since
> they allow you to basically push a set of packages which should be
> installed and control "per type of vm" use flags and package keywords
> (eg I have www-nginx, www-apache, mail, proxy, mysql, ftp, etc
> profiles).  Additionally I have a small overlay of local ebuilds that
> sit in the same tree
>

I'd rather not maintain a separate profile for each server/node type.
Instead it would seem to be easier to use to a buildhost to create a combined 
repository of packages, and then use puppet to define nodetypes and tell each 
individual node which packages to install, which users to create, and which 
services to enable.

> Up until now I haven't really made any effort to sync this whole tree
> across multiple physical machines and it's a bit of an ad-hoc process.
> Using something like git would probably be perfect
>

Agreed -- git *is* perfect and widely used in enterprise configuration 
management repositories.  The main advantage is that it allows individual 
users to maintain their own forks, and yet merge contributions from other 
trees or to request that other repositories merge their changes w/o much 
effort.  Github makes this even easier w/ a web interface, tools to visual 
changes and track contributions, and discussions/wiki's for each repository.

> The still missing step is configuration management across the machine
> types, eg I want to upgrade all my "Apache-WWW" class machines and merge
> in all changes in /etc in a certain way... At the moment I just run
> dispatch-conf across all machines, but it can be quite boring merging 20
> instances of sshd.conf...  Seems like Puppet/Chef could be a solution
> here, but the step up and investment to make it work seems pretty large?
>

Yes, I'm currently planning that puppet will control installation of new 
packages and pushing all configuration files.  This works very well w/ RedHat 
Enterprise Linux/etc and if we are careful should also work for gentoo-- For 
updates, we may have to either extend puppet to be aware of revdep-rebuild and 
gentoo package versioning better, or have an external script for updating the 
relevant nodes after the buildhost has finished compiling them.

>
>
> It does appear like managing large numbers of virtual machines is one
> are that gentoo could score very well?  Interested to see any chatter on
> how others solve this problem, or any general advocacy?  Probably we
> should start a new thread though...
>

Agreed - With careful management, the time investment in QA and package 
management/maintenance should be more than outweighed by the benefits for 
reasonably sized server clusters.  It would be even easier if we can have 
gentoo server admins work together to help build puppet modules and general 
mgmt utilities using a service like github.

> Regards
>
> Ed W

Matt
-- 
Matthew Marlowe    /  858-400-7430  /    DeployLinux Consulting, Inc
  Professional Linux Hosting and Systems Administration Services
              www.deploylinux.net   *   matt@×××××××××××.net
                             'MattM' @ irc.freenode.net