1 |
Ian P. Christian wrote: |
2 |
|
3 |
> I would like to move to an automated way of keeping these servers up to |
4 |
> date. |
5 |
> |
6 |
> Firstly, I would like to use binary packages. I initially thought about |
7 |
> having a repository pre architecture, and standardising on cflags and use |
8 |
> flags - however this isn't feasable if you need to alter the use flags on |
9 |
> just one server - installing from the binary package will result in the |
10 |
> wrong package being installed. |
11 |
|
12 |
I use a central server to build and host all binary packages that are used |
13 |
to install the various server types. |
14 |
|
15 |
In a nutshell, this is how it works. |
16 |
|
17 |
One hoss server contains chroot build hosts for every individual server type |
18 |
in my farm (web servers, dns servers, mail exchangers, database servers, |
19 |
etc...) This server does all of the compiling of binary packages for each |
20 |
arch it can support (pentium4 or below). |
21 |
|
22 |
A "build host" logically consists of a specific portage snapshot and a |
23 |
specific set of config files (make.conf, /etc/portage/*, etc..). |
24 |
|
25 |
Each individual build host contains a flag file - /etc/conf.d/buildhost, |
26 |
which is used to store the snapshot version and as a method to change the |
27 |
behavior during the emerging of certain packages (kernel, config files), |
28 |
etc.. |
29 |
|
30 |
Each portage snapshot (local and overlays) is shared via rsync from the |
31 |
central server. GENTOO_MIRRORS on the target servers syncs from there. |
32 |
|
33 |
Each build host /usr/portage/packages directory is symlinked into the apache |
34 |
document root on the central server. PORTAGE_BINHOST on the target servers |
35 |
points there. |
36 |
|
37 |
An example directory structure looks like this: |
38 |
|
39 |
/buildhosts/stage3-p3 -> chroot environment to build a pentium3 stage3 |
40 |
/buildhosts/stage3-p4 -> chroot environment to build a pentium4 stage3 |
41 |
/buildhosts/web-servers -> chroot environment to build web servers |
42 |
/buildhosts/dns-servers -> chroot environment to build dns servers |
43 |
/buildhosts/mysql-servers -> chroot environment to build mysql servers |
44 |
|
45 |
Each server type uses a stage3 generated from the corresponding stage3 build |
46 |
host. The portage directory within each buildhost is bind mounted to the |
47 |
corresponding portage snapshot |
48 |
in /buildhosts/shared/snapshots/YYYYMMDD/portage (see below). Each chroot |
49 |
build host has /dev and /proc bind mounted. |
50 |
|
51 |
/buildhosts/shared/snapshots/20051112/portage -> portage snapshot |
52 |
/buildhosts/shared/snapshots/20060201/portage -> portage snapshot |
53 |
|
54 |
These are the actual portage snapshots, extracted, from the specified date. |
55 |
These directories will be bind mounted within the chroot build hosts that |
56 |
use them. |
57 |
|
58 |
/buildhosts/shared/configs/stage3-p3/20060201 |
59 |
/buildhosts/shared/configs/stage3-p4/20060201 |
60 |
/buildhosts/shared/configs/web-servers/20060201 |
61 |
/buildhosts/shared/configs/dns-servers/20051112 |
62 |
|
63 |
Config files for each build host. I use a script that sources the buildhost |
64 |
etc/conf.d/buildhost flag file, determines the value of SNAPSHOT_DATE, and |
65 |
if CONFIGS_COPIED is not defined it will copy everything over to the chroot |
66 |
environment from the corresponding config directory. This is how each |
67 |
build host gets its needed make.conf, resolv.conf, /etc/portage/*, etc... |
68 |
|
69 |
/buildhosts/shared/distfiles |
70 |
|
71 |
This is bind mounted to each chroot build host /usr/portage/distfiles so |
72 |
that a common source of distfiles can be used. |
73 |
|
74 |
/buildhosts/shared/packages/stage3-p3/20060201 |
75 |
/buildhosts/shared/packages/web-servers/20060201 |
76 |
/buildhosts/shared/packages/dns-servers/20051112 |
77 |
|
78 |
Each of these is bind mounted to the corresponding chroot build |
79 |
host /usr/portage/packages directory. When the binary packages are built |
80 |
they end up here. These are the directories that will be symlinked into |
81 |
the apache document tree and obtained on the target servers via the |
82 |
PORTAGE_BINHOST variable... |
83 |
|
84 |
/buildhosts/shared/overlays/web-servers |
85 |
/buildhosts/shared/overlays/dns-servers |
86 |
|
87 |
Where I keep portage overlays that are bind mounted to the corresponding |
88 |
chroot build host. I use these overlay trees to create kernel ebuilds, |
89 |
config file ebuilds, web packages, dns zone file packages, etc... I keep |
90 |
track of these with the etc/conf.d/buildhost OVERLAYS variable. |
91 |
|
92 |
/buildhosts/shared/stages |
93 |
|
94 |
Where I keep stage1's and custom stage tarballs that are extracted if STAGE |
95 |
is set in etc/conf.d/buildhost. |
96 |
|
97 |
So basically, all emerges are done within each chroot build host. The |
98 |
resulting portage snapshot and binary packages wind up being available via |
99 |
rsync and apache to the servers to be installed. |
100 |
|
101 |
Each server type (web-servers, dns-servers, etc..) gets its own meta package |
102 |
via the overlays that determines what apps are compiled. |
103 |
|
104 |
Each individual server gets its own unique config file package within the |
105 |
overlays also. These config packages work much like the baselayout |
106 |
packages, the actual files are installed |
107 |
to /usr/share/server-configs/web-servers/20060201, in pkg_postinst() if |
108 |
there is no /etc/conf.d/buildhost, a cp -r is done that copies the config |
109 |
files over into the / filesystem.... |
110 |
|
111 |
Got all that :) |
112 |
|
113 |
-- |
114 |
gentoo-server@g.o mailing list |