Gentoo Logo
Gentoo Spaceship

Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-dev
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
To: gentoo-dev@g.o
From: Enrico Weigelt <weigelt@...>
Subject: Re: avoiding urgent stabilizations
Date: Fri, 25 Feb 2011 18:53:10 +0100
* Ed W <lists@...> schrieb:

> I maintain a, likely much smaller, number of VMs using linux vservers.  
> The approach here is to almost cut each machine down to a chroot that 
> runs only one (or thereabouts) interesting service.

I'm working in a similar way: my dedicated boxes are VM hosts
(currently ovz, but later lguest or lxc), each of them running
only specific services (eg. one for nginx-based front proxy,
several other for backend webservers, RDBMs'es, mailservers, etc).

But, for me, even a trimmed-down Gentoo is still too large
(has to contain the whole base packages, from portage to
toolchain, includes, etc). I'd prefer having only the essential
runtime stuff within the containers.

For this we need a different approach (strictly separating build
and production environments). Binary distros (eg. Debian) might
be one option, but they're lacking the configurability and mostly
are still too large. So I'm going a different route using my own
buildsystem - called Briegel - which originally was designed for
embedded/small-device targets.

For now I didn't have the spare time to port all the packages
required for complete server systems (most of it is making
them all cleanly crosscompile'able, as this is a fundamental
concept of Briegel). But maybe you'd like to join in and try it :)

> Using something like git would probably be perfect

I'm using git'ed portage trees in production for several month now
(I sometimes had to touch some eclasses, which _IMHO_ doesn't via
overlays). The repos are configured to rebase on pull and everything
runs automatically :)

> The still missing step is configuration management across the machine 
> types, eg I want to upgrade all my "Apache-WWW" class machines and merge 
> in all changes in /etc in a certain way... At the moment I just run 
> dispatch-conf across all machines, but it can be quite boring merging 20 
> instances of sshd.conf...  Seems like Puppet/Chef could be a solution 
> here, but the step up and investment to make it work seems pretty large?

I haven't used puppet yet, but several collegues have made good
experiences with it. If you're maintaining dozens of very similar
systems (mine tend to be very different from each other), it likely
worth investigating.

> It does appear like managing large numbers of virtual machines is one 
> are that gentoo could score very well?  Interested to see any chatter on 
> how others solve this problem, or any general advocacy?  Probably we 
> should start a new thread though...

I'm not sure if Gentoo really is the right distro for that purpose,
as it's targeted to very different systems (i.g. Gentoo boxes are
expected to be quite unique, beginning with different per-package
useflags, even down to cflags, etc). But it might still be a good
basis for building specific system images (let's call them stage5 ;-))

An setup for 100 equal webserver vm's could look like this:

* run a normal Gentoo vm (tailored for the webserver appliance),
  where do you do regular updates (emerge, revdep-rebuild, etc, etc)
* from time to time take a snapshot, strip off the buildtime-only
  stuff (hmm, could turn out to be a bit tricky ;-o)
* this stripped snapshot now goes into testing vm's
* when approved, the individual production vm's are switched over
  to the new image (maybe using some mount magic, unionfs, etc)

At this point I've got a question for to the other folks here:

emerge has an --root option which allows to (un)merge in a separate
system image. So it should be possible to unmerge a lot of system
packages which are just required for updating/building (even
portage itself), but this still will be manual - what about 
dependency handling ?

Is there some way to drop at least parts of the standard system set,
so eg. portage, python, gcc, etc, etc get unmerged by --depclean
if nobody else (in world set) doesn't explicitly require them ?

 Enrico Weigelt, metux IT service --

 phone:  +49 36207 519931  email: weigelt@...
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme

Re: avoiding urgent stabilizations
-- Ed W
avoiding urgent stabilizations
-- PaweĊ‚ Hajdan, Jr.
Re: avoiding urgent stabilizations
-- Enrico Weigelt
Re: avoiding urgent stabilizations
-- Ed W
Re: avoiding urgent stabilizations
-- Matthew Marlowe
Re: avoiding urgent stabilizations
-- Ed W
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Community Development of Gentoo Server Mgmt Tools for VMware Clusters/etc - Puppet modules, github collaboration, etc - Fork of Dev Topic: Avoiding Urgent Stabilizations
Next by thread:
Re: avoiding urgent stabilizations
Previous by date:
Last Rites: media-gfx/pornview
Next by date:
Re: Resigning from apache herd

Updated Jun 29, 2012

Summary: Archive of the gentoo-dev mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.