Gentoo Archives: gentoo-dev

From: Enrico Weigelt <weigelt@×××××.de>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] avoiding urgent stabilizations
Date: Sat, 26 Feb 2011 16:26:13
Message-Id: 20110226155750.GH22774@nibiru.local
In Reply to: Re: [gentoo-dev] avoiding urgent stabilizations by Ed W
1 * Ed W <lists@××××××××××.com> schrieb:
2
3 > I'm just building some embedded devices on the side using
4 > gentoo and my minimal builds are only a few MB?
5
6 How to do you get out all the buildtime stuff (portage, toolchain, etc) ?
7
8 > Seems like your complaint is that you have gentoo installs which are
9 > full featured with a toolchain and portage, which you are comparing to
10 > an installation you built with a different tool that doesn't have a
11 > toolchain installed?
12
13 Well, with Briegel, I'm strictly differenciating between the build
14 host and the target image. Everything's crosscompiled by design.
15 So I can explicitly define which packages need to be in the final
16 image (and only those are built). The build host can by any fairly
17 recent GNU/Linux system (happens to be Gentoo at my site)
18
19 > 1) Build your base images without a toolchain or portage and use a
20 > minimal package installer to install pre-built binary packages.
21 > This seems fraught with issues long term though...
22
23 How can this be done w/ Gentoo ?
24 AFAIK it always has a system set containing things like portage,
25 toolchain, etc, etc.
26
27 > 2) Build your base images without a toolchain, but with portage (and
28 > perhaps a very minimal python). This gives you full dependency tracking
29 > and obviously bind mount/nfs mount the actual portage tree to avoid
30 > space used there. This seems workable and minimal?
31
32 Would be still too much for my goals: I don't wanna have anything in
33 the final system that's not necessary for runtime. Building and even
34 installation should be driven from outside (actually, the final system
35 could even have /bin, /usr, etc mounted ro and /var w/o noexec)
36
37 > 3) If we are talking virtual machines then who cares if your containers
38 > are individually quite large, if the files in them are duplicated across
39 > all containers? Simply use an appropriate de-duplication strategy to
40 > coalesce the space and most of the disadvantages disappear?
41
42 Well, that might be an option, if the *is* enough duplication
43 (within an storage enclousure). But storage space is not the only
44 reason. My personal background is from medical and industrial
45 appliances, so I have some more rigid rules ;-o
46
47 > Personally I think option 3) is quite interesting from the medium number
48 > of virtual machines, ie in the 10s to hundreds, ie simply don't worry
49 > about it, let the OS do the work. In the hundreds to thousands plus
50 > level I guess you have unique challenges and I would be wrong to try and
51 > suggest a solution from the comfort of a laptop without having that
52 > responsibility, but I would have thought there was some advantage in a
53 > very rigidly deployed base OS generated and updated very precisely?
54
55 I'm thinking of several thousands of vm's for dedicated purposes,
56 which fall under similar requirements as industrial embedded
57 systems (eg. there will be practically no human operator).
58
59 > >For this we need a different approach (strictly separating build
60 > >and production environments). Binary distros (eg. Debian) might
61 > >be one option, but they're lacking the configurability and mostly
62 > >are still too large. So I'm going a different route using my own
63 > >buildsystem - called Briegel - which originally was designed for
64 > >embedded/small-device targets.
65 > >
66 > >For now I didn't have the spare time to port all the packages
67 > >required for complete server systems (most of it is making
68 > >them all cleanly crosscompile'able, as this is a fundamental
69 > >concept of Briegel). But maybe you'd like to join in and try it :)
70 >
71 > Sounds like an interesting challenge, but I'm unconvinced you can't
72 > solve 90% of your problem within the constraints of Gentoo? This saves
73 > you a bunch of time that could be invested in the last 10% through more
74 > traditional means?
75
76 Maybe. But I need 100%. (cant explain all the reasons here right now).
77 I'm well aware that this all is far out of Gentoo's scope.
78
79 > >I'm not sure if Gentoo really is the right distro for that purpose,
80 > >as it's targeted to very different systems (i.g. Gentoo boxes are
81 > >expected to be quite unique, beginning with different per-package
82 > >useflags, even down to cflags, etc). But it might still be a good
83 > >basis for building specific system images (let's call them stage5 ;-))
84 >
85 > I won't disagree on your "where it's targeted", but just to re-iterate
86 > why I think Gentoo works well is that it does have a very workable
87 > binary package feature!
88
89 Note that Gentoo's binary packages (which are practically snapshots
90 of what will be copied in by merge stage) are alway bound to specific
91 domains, eg. the full configuration (make.conf, useflags, etc, etc),
92 and there may be a lot of interdependencies between installed packages
93 (and versions), so it takes some care to use it in stable ways.
94
95 This might be very fine for you (eg. if you have dozens of really
96 equal systems), but you'll have to be aware of the limitations.
97
98 > My way of working is to use (several) shared binary package repos and
99 > the guests largely pull from those shared package directories. In fact
100 > what I do is have a minimal number of shared "/usr/portage/package"
101 > directories and I mount an appropriate one to the guest type at boot
102 > time. At the moment my main two options are "32bit" and "64bit" for the
103 > package mounts, but I recently introduced a new machine type which is
104 > held back to perl 5.8 and that guest gets it's own package mount since
105 > it's obviously linking a lot of binaries differently
106
107 As long as you have the same make.conf/useflag/etc settings for all
108 systems (within a guest type), this might work well.
109
110 > Now, the icing is that this works extremely well even once you decide to
111 > lightly customise machine types. So for example my binary packages are
112 > very high level (eg 32/64bit), my "profiles" would be fairly high level,
113 > eg I have www-apache and www-nginx base profiles. However, a specific
114 > virtual machine running say nginx might itself need a specific PHP
115 > application installed, and that itself might need some dependencies,
116 > which in turn might require a specific set of customisation of use flags
117 > and versions.
118
119 At this point, Gentoo's binary packages could become tricky.
120 The big questions eg. are: can it handle multiple binpkgs of the
121 same (source) package with differing useflags ? How are version-
122 specific dependencies handled, etc, etc.
123
124 > >An setup for 100 equal webserver vm's could look like this:
125 > >
126 > >* run a normal Gentoo vm (tailored for the webserver appliance),
127 > > where do you do regular updates (emerge, revdep-rebuild, etc, etc)
128 > >* from time to time take a snapshot, strip off the buildtime-only
129 > > stuff (hmm, could turn out to be a bit tricky ;-o)
130 > >* this stripped snapshot now goes into testing vm's
131 > >* when approved, the individual production vm's are switched over
132 > > to the new image (maybe using some mount magic, unionfs, etc)
133 >
134 > This could work and perhaps for 100 identical Vms you have enough meat
135 > to work on something quite customised anyway?
136
137 Yes, in this case, all the VMs have to have identical build configuration.
138 Once you need things like differing useflag, that approach wont fit anymore.
139
140 > >At this point I've got a question for to the other folks here:
141 > >
142 > >emerge has an --root option which allows to (un)merge in a separate
143 > >system image. So it should be possible to unmerge a lot of system
144 > >packages which are just required for updating/building (even
145 > >portage itself), but this still will be manual - what about
146 > >dependency handling ?
147 >
148 > This is correct. In fact this is how you build a stage 1,2,3 etc and
149 > how catalyst works!
150 >
151 > The information is a bit spread out over several out of date wiki
152 > articles, but perhaps start with:
153 > http://en.gentoo-wiki.com/wiki/Tiny_Gentoo
154 >
155 > Roughly speaking you could "freshen" your current installation with
156 > (from memory):
157 > ROOT="/tmp/new_build" emerge -av world
158
159 Is this just the installation root or also the sysroot for the
160 toolchain ?
161
162 <snip>
163
164 > Further, if you wish you can make those VMs have a reduced or missing
165 > toolchain etc.
166
167 hmm, do I have to unmerge certain packages explicitly or can I
168 tell it to only install those packages required for runtime in
169 the first place ?
170
171 > >Is there some way to drop at least parts of the standard system set,
172 > >so eg. portage, python, gcc, etc, etc get unmerged by --depclean
173 > >if nobody else (in world set) doesn't explicitly require them ?
174 >
175 > You are almost thinking about it all wrong. ("There is no spoon...")
176 >
177 > This is gentoo, so at this more advanced level, stop thinking about
178 > "standard system set" and instead free your mind to start with
179 > "nothing".
180
181 With "standard system set" I mean those packages which are in @system
182 on a particular arch. In my vision, @system would be simply empty,
183 instead @world tells which packages you want and the dependencies
184 are pulled in automatically.
185
186 No idea if that's valid anymore, but some (long?) time ago, there
187 was the rule that individual packages weren't allowed to explicitly
188 DEPEND= on system packages. Obviously this wouldn't work with an
189 empty system set (you'd have to find out the deps and add them
190 to world set manually).
191
192 > Largely the portage dependency tracker will help you pull in the minimal
193 > needed dependencies, but beware that system packages arent generally
194 > explicitly tracked so you may stumble across some deps when you are
195 > going really basic and omiting standard system packages (just use common
196 > sense: it should be fairly obvious if an application requires a compiler
197 > and you didn't install one then you have a conflict of interest...)
198
199 Well, that's where it gets complicated: if some packages depend on some
200 library libfoo in @system (which I would empty in this case), portage
201 lacks information to compute the full dependency tree, so I'd have to
202 find them out and add them to @world manually. Exactly what I do not want.
203
204
205 OTOH, Briegel doenst have that issue, as all dependencies are stated
206 explicitly in each package (well, with one exception: libc).
207 (of course, Briegel yet lacks a large package repository as I'm
208 still working mostly alone on it)
209
210
211 cu
212 --
213 ----------------------------------------------------------------------
214 Enrico Weigelt, metux IT service -- http://www.metux.de/
215
216 phone: +49 36207 519931 email: weigelt@×××××.de
217 mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
218 ----------------------------------------------------------------------
219 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
220 ----------------------------------------------------------------------

Replies

Subject Author
Re: [gentoo-dev] avoiding urgent stabilizations Ed W <lists@××××××××××.com>