| 1 |
Ian P. Christian wrote: |
| 2 |
|
| 3 |
> I would like to move to an automated way of keeping these servers up to |
| 4 |
> date. |
| 5 |
> |
| 6 |
> Firstly, I would like to use binary packages. I initially thought about |
| 7 |
> having a repository pre architecture, and standardising on cflags and use |
| 8 |
> flags - however this isn't feasable if you need to alter the use flags on |
| 9 |
> just one server - installing from the binary package will result in the |
| 10 |
> wrong package being installed. |
| 11 |
|
| 12 |
I use a central server to build and host all binary packages that are used |
| 13 |
to install the various server types. |
| 14 |
|
| 15 |
In a nutshell, this is how it works. |
| 16 |
|
| 17 |
One hoss server contains chroot build hosts for every individual server type |
| 18 |
in my farm (web servers, dns servers, mail exchangers, database servers, |
| 19 |
etc...) This server does all of the compiling of binary packages for each |
| 20 |
arch it can support (pentium4 or below). |
| 21 |
|
| 22 |
A "build host" logically consists of a specific portage snapshot and a |
| 23 |
specific set of config files (make.conf, /etc/portage/*, etc..). |
| 24 |
|
| 25 |
Each individual build host contains a flag file - /etc/conf.d/buildhost, |
| 26 |
which is used to store the snapshot version and as a method to change the |
| 27 |
behavior during the emerging of certain packages (kernel, config files), |
| 28 |
etc.. |
| 29 |
|
| 30 |
Each portage snapshot (local and overlays) is shared via rsync from the |
| 31 |
central server. GENTOO_MIRRORS on the target servers syncs from there. |
| 32 |
|
| 33 |
Each build host /usr/portage/packages directory is symlinked into the apache |
| 34 |
document root on the central server. PORTAGE_BINHOST on the target servers |
| 35 |
points there. |
| 36 |
|
| 37 |
An example directory structure looks like this: |
| 38 |
|
| 39 |
/buildhosts/stage3-p3 -> chroot environment to build a pentium3 stage3 |
| 40 |
/buildhosts/stage3-p4 -> chroot environment to build a pentium4 stage3 |
| 41 |
/buildhosts/web-servers -> chroot environment to build web servers |
| 42 |
/buildhosts/dns-servers -> chroot environment to build dns servers |
| 43 |
/buildhosts/mysql-servers -> chroot environment to build mysql servers |
| 44 |
|
| 45 |
Each server type uses a stage3 generated from the corresponding stage3 build |
| 46 |
host. The portage directory within each buildhost is bind mounted to the |
| 47 |
corresponding portage snapshot |
| 48 |
in /buildhosts/shared/snapshots/YYYYMMDD/portage (see below). Each chroot |
| 49 |
build host has /dev and /proc bind mounted. |
| 50 |
|
| 51 |
/buildhosts/shared/snapshots/20051112/portage -> portage snapshot |
| 52 |
/buildhosts/shared/snapshots/20060201/portage -> portage snapshot |
| 53 |
|
| 54 |
These are the actual portage snapshots, extracted, from the specified date. |
| 55 |
These directories will be bind mounted within the chroot build hosts that |
| 56 |
use them. |
| 57 |
|
| 58 |
/buildhosts/shared/configs/stage3-p3/20060201 |
| 59 |
/buildhosts/shared/configs/stage3-p4/20060201 |
| 60 |
/buildhosts/shared/configs/web-servers/20060201 |
| 61 |
/buildhosts/shared/configs/dns-servers/20051112 |
| 62 |
|
| 63 |
Config files for each build host. I use a script that sources the buildhost |
| 64 |
etc/conf.d/buildhost flag file, determines the value of SNAPSHOT_DATE, and |
| 65 |
if CONFIGS_COPIED is not defined it will copy everything over to the chroot |
| 66 |
environment from the corresponding config directory. This is how each |
| 67 |
build host gets its needed make.conf, resolv.conf, /etc/portage/*, etc... |
| 68 |
|
| 69 |
/buildhosts/shared/distfiles |
| 70 |
|
| 71 |
This is bind mounted to each chroot build host /usr/portage/distfiles so |
| 72 |
that a common source of distfiles can be used. |
| 73 |
|
| 74 |
/buildhosts/shared/packages/stage3-p3/20060201 |
| 75 |
/buildhosts/shared/packages/web-servers/20060201 |
| 76 |
/buildhosts/shared/packages/dns-servers/20051112 |
| 77 |
|
| 78 |
Each of these is bind mounted to the corresponding chroot build |
| 79 |
host /usr/portage/packages directory. When the binary packages are built |
| 80 |
they end up here. These are the directories that will be symlinked into |
| 81 |
the apache document tree and obtained on the target servers via the |
| 82 |
PORTAGE_BINHOST variable... |
| 83 |
|
| 84 |
/buildhosts/shared/overlays/web-servers |
| 85 |
/buildhosts/shared/overlays/dns-servers |
| 86 |
|
| 87 |
Where I keep portage overlays that are bind mounted to the corresponding |
| 88 |
chroot build host. I use these overlay trees to create kernel ebuilds, |
| 89 |
config file ebuilds, web packages, dns zone file packages, etc... I keep |
| 90 |
track of these with the etc/conf.d/buildhost OVERLAYS variable. |
| 91 |
|
| 92 |
/buildhosts/shared/stages |
| 93 |
|
| 94 |
Where I keep stage1's and custom stage tarballs that are extracted if STAGE |
| 95 |
is set in etc/conf.d/buildhost. |
| 96 |
|
| 97 |
So basically, all emerges are done within each chroot build host. The |
| 98 |
resulting portage snapshot and binary packages wind up being available via |
| 99 |
rsync and apache to the servers to be installed. |
| 100 |
|
| 101 |
Each server type (web-servers, dns-servers, etc..) gets its own meta package |
| 102 |
via the overlays that determines what apps are compiled. |
| 103 |
|
| 104 |
Each individual server gets its own unique config file package within the |
| 105 |
overlays also. These config packages work much like the baselayout |
| 106 |
packages, the actual files are installed |
| 107 |
to /usr/share/server-configs/web-servers/20060201, in pkg_postinst() if |
| 108 |
there is no /etc/conf.d/buildhost, a cp -r is done that copies the config |
| 109 |
files over into the / filesystem.... |
| 110 |
|
| 111 |
Got all that :) |
| 112 |
|
| 113 |
-- |
| 114 |
gentoo-server@g.o mailing list |
Archives