1 |
On 26/02/2011 15:57, Enrico Weigelt wrote: |
2 |
> * Ed W<lists@××××××××××.com> schrieb: |
3 |
> |
4 |
>> I'm just building some embedded devices on the side using |
5 |
>> gentoo and my minimal builds are only a few MB? |
6 |
> How to do you get out all the buildtime stuff (portage, toolchain, etc) ? |
7 |
> |
8 |
>> Seems like your complaint is that you have gentoo installs which are |
9 |
>> full featured with a toolchain and portage, which you are comparing to |
10 |
>> an installation you built with a different tool that doesn't have a |
11 |
>> toolchain installed? |
12 |
> Well, with Briegel, I'm strictly differenciating between the build |
13 |
> host and the target image. Everything's crosscompiled by design. |
14 |
> So I can explicitly define which packages need to be in the final |
15 |
> image (and only those are built). The build host can by any fairly |
16 |
> recent GNU/Linux system (happens to be Gentoo at my site) |
17 |
|
18 |
I think you are missing the point that the gentoo build system is quite |
19 |
happily building you a destination build with as little as one package |
20 |
in it? Whether that destination boots is a function of careful package |
21 |
choice, but the point is you are definitely not constrained from |
22 |
building a highly minimal target if you wish? |
23 |
|
24 |
Just from the top of my head I should imagine this would likely boot: |
25 |
|
26 |
mkdir -p /newroot/{proc,dev,sys} |
27 |
cp -a /dev/{null,zero} /newroot/dev/ |
28 |
emerge —root /newroot glibc shadow baselayout openrc util-linux udev |
29 |
|
30 |
|
31 |
If you want to go more minimal then I have a uclibc chroot that I use as |
32 |
a build environment, the first line of my build script for my minimal |
33 |
embedded device is simply: |
34 |
|
35 |
emerge --root /embedded baselayout uclibc busybox |
36 |
|
37 |
I haven't checked, but give or take creating /proc, /sys and some |
38 |
minimal devices in /dev, the above should boot fine. |
39 |
|
40 |
|
41 |
|
42 |
If you were booting into a "container" based virtualisation then you |
43 |
don't need to manage /dev and mounting devices and so you can get even |
44 |
more minimal than the top example I posted (ie I don't think you need |
45 |
udev, etc) |
46 |
|
47 |
Obviously such minimal builds will have minimal package management, but |
48 |
this may not be a problem if you have a super fixed configuration? |
49 |
Simply construct your environment such that the data is separate from |
50 |
the OS and also make sure any OS customisation is scriptable and now you |
51 |
would have a way to occasionally build a new image which you rolled out |
52 |
to all virtual machines |
53 |
|
54 |
|
55 |
>> 1) Build your base images without a toolchain or portage and use a |
56 |
>> minimal package installer to install pre-built binary packages. |
57 |
>> This seems fraught with issues long term though... |
58 |
> How can this be done w/ Gentoo ? |
59 |
> AFAIK it always has a system set containing things like portage, |
60 |
> toolchain, etc, etc. |
61 |
|
62 |
"System" is just a statement of some defaults. At the level you are |
63 |
working at you can largely ignore it. |
64 |
|
65 |
Build what you like into the target build.... |
66 |
|
67 |
|
68 |
> Would be still too much for my goals: I don't wanna have anything in |
69 |
> the final system that's not necessary for runtime. Building and even |
70 |
> installation should be driven from outside (actually, the final system |
71 |
> could even have /bin, /usr, etc mounted ro and /var w/o noexec) |
72 |
|
73 |
OK, so sounds like a simple build script like the one above would suit you? |
74 |
|
75 |
For more advanced use consider learning to use catalyst or Funtoos build |
76 |
system? These are full on scriptable ways to repeatably build non |
77 |
trivial target builds. (remember they can build as little as single |
78 |
package targets or more complicated builds) |
79 |
|
80 |
|
81 |
> Note that Gentoo's binary packages (which are practically snapshots |
82 |
> of what will be copied in by merge stage) are alway bound to specific |
83 |
> domains, eg. the full configuration (make.conf, useflags, etc, etc), |
84 |
> and there may be a lot of interdependencies between installed packages |
85 |
> (and versions), so it takes some care to use it in stable ways. |
86 |
> |
87 |
> This might be very fine for you (eg. if you have dozens of really |
88 |
> equal systems), but you'll have to be aware of the limitations. |
89 |
|
90 |
Well, your requirements suggest that you don't need a package manager |
91 |
and you will instead build a new read-only OS build every so often and |
92 |
deploy that. However, in the interests of exploring the idea further: |
93 |
|
94 |
- By definition binary packages only work where you have equal systems? |
95 |
This is true of gentoo AND rpm/deb. |
96 |
- If you need binary packages for non equal systems then you need some |
97 |
kind of namespacing method to separate the binaries. Gentoo has no |
98 |
automated way to do this, but it's reasonably workable to have binary |
99 |
package repos and simply tell your guest machine to use the appropriate |
100 |
binary repo |
101 |
- Gentoo's binaries are slightly cleverer than some notice in that they |
102 |
have some awareness of when they don't match the target and the operator |
103 |
has the option to either ignore that and continue the install, OR to |
104 |
ignore the binary package and generate it's own binaries using the local |
105 |
compiler, etc. |
106 |
|
107 |
So, I don't really see the problem, in fact Gentoo binaries can be seen |
108 |
as slightly more flexible than RPMs to some extent if you allow them to |
109 |
be "optional". |
110 |
|
111 |
|
112 |
> As long as you have the same make.conf/useflag/etc settings for all |
113 |
> systems (within a guest type), this might work well. |
114 |
|
115 |
But that's true of Redhat/Debian/Gentoo? |
116 |
|
117 |
RPMs don't allow you to have a single binary repo where some of the |
118 |
packages are linked against perl 5.8 and some against 5.12? I guess it |
119 |
would be possible to enhance the packaging format to cater for that, but |
120 |
largely the whole premise of binary packages is that you lock down your |
121 |
target options and build all the systems identically? There is no easy |
122 |
way around this? (In fact it's what you want) |
123 |
|
124 |
|
125 |
> At this point, Gentoo's binary packages could become tricky. |
126 |
> The big questions eg. are: can it handle multiple binpkgs of the |
127 |
> same (source) package with differing useflags ? How are version- |
128 |
> specific dependencies handled, etc, etc. |
129 |
|
130 |
As it stands, no it can't. However, neither can RPM/DEB... |
131 |
|
132 |
Remember the premise of binaries is that they are all the same... Pick a |
133 |
system configuration and build all your binaries with that |
134 |
configuration. If you need to support multiple configurations then setup |
135 |
multiple binary repositories |
136 |
|
137 |
>> articles, but perhaps start with: |
138 |
>> http://en.gentoo-wiki.com/wiki/Tiny_Gentoo |
139 |
>> |
140 |
>> Roughly speaking you could "freshen" your current installation with |
141 |
>> (from memory): |
142 |
>> ROOT="/tmp/new_build" emerge -av world |
143 |
> Is this just the installation root or also the sysroot for the |
144 |
> toolchain ? |
145 |
|
146 |
I think you need to read through the tiny_gentoo type wiki pages. The |
147 |
general build process is: |
148 |
|
149 |
1) Start with a working machine |
150 |
2) Build a build environment |
151 |
3) Jump into the build environment and repeat step 2) to build a target |
152 |
environment |
153 |
4) Jump into the target environment |
154 |
|
155 |
Now in the absence of cross compiling you can often skip step 2) and |
156 |
build the target environment directly, but it's normally useful to have |
157 |
that build environment so that you can customise things a bit. |
158 |
|
159 |
Also this will help explain some of the catalyst recipes and why you see |
160 |
everything built twice, once to get a build environment, then again to |
161 |
build the target build |
162 |
|
163 |
|
164 |
>> Further, if you wish you can make those VMs have a reduced or missing |
165 |
>> toolchain etc. |
166 |
> hmm, do I have to unmerge certain packages explicitly or can I |
167 |
> tell it to only install those packages required for runtime in |
168 |
> the first place ? |
169 |
|
170 |
Well your really simplest "build" would be to take a stage3/4 and simply |
171 |
"unmerge" anything you don't need... bit crude though |
172 |
|
173 |
However, from the email above you can see that it's more desirable to |
174 |
instead "build up", starting with an empty build and from there add the |
175 |
packages you actually want. |
176 |
|
177 |
|
178 |
>> You are almost thinking about it all wrong. ("There is no spoon...") |
179 |
>> |
180 |
>> This is gentoo, so at this more advanced level, stop thinking about |
181 |
>> "standard system set" and instead free your mind to start with |
182 |
>> "nothing". |
183 |
> With "standard system set" I mean those packages which are in @system |
184 |
> on a particular arch. In my vision, @system would be simply empty, |
185 |
> instead @world tells which packages you want and the dependencies |
186 |
> are pulled in automatically. |
187 |
|
188 |
Seriously, "there is no spoon"... |
189 |
|
190 |
If you want to make your own build script then there isn't even an |
191 |
@system... |
192 |
|
193 |
> No idea if that's valid anymore, but some (long?) time ago, there |
194 |
> was the rule that individual packages weren't allowed to explicitly |
195 |
> DEPEND= on system packages. Obviously this wouldn't work with an |
196 |
> empty system set (you'd have to find out the deps and add them |
197 |
> to world set manually). |
198 |
|
199 |
I believe this is correct. However, there aren't many packages in system |
200 |
and so whilst it's kind of buyer beware once you start chopping stuff |
201 |
out of system, the point is only if you don't install gzip or gcc then |
202 |
you need to beware stuff which depends on either. |
203 |
|
204 |
I think in practice manually managing those dependencies is going to be |
205 |
very straightforward, ie you are more going to fix the applications to |
206 |
not require the dependencies. After all if you don't want gcc, then you |
207 |
really don't want gcc, simple really... eg, if some simple application |
208 |
uses make to update it's config files then you will likely look at some |
209 |
workaround rather than installing make... |
210 |
|
211 |
|
212 |
> Well, that's where it gets complicated: if some packages depend on some |
213 |
> library libfoo in @system (which I would empty in this case), portage |
214 |
> lacks information to compute the full dependency tree, so I'd have to |
215 |
> find them out and add them to @world manually. Exactly what I do not want. |
216 |
|
217 |
Oh come on. You are working at the level of building your OS image from |
218 |
scratch and you don't know if any of your applications require (at |
219 |
runtime) gcc, bison, m4, wget, man, os-headers, portage or ssh? |
220 |
|
221 |
It seems likely that you can quickly determine if you can live without |
222 |
most of the above (at runtime)...?!! |
223 |
|
224 |
> OTOH, Briegel doenst have that issue, as all dependencies are stated |
225 |
> explicitly in each package (well, with one exception: libc). |
226 |
|
227 |
So, you can probably already use that information to figure out your |
228 |
minimal set of system packages that you likely need (for runtime). |
229 |
|
230 |
Just checking that Briegel tracks both runtime and build time |
231 |
dependencies separately? eg you might need gzip to build something, but |
232 |
not to run it? |
233 |
|
234 |
|
235 |
So far I think you could easily convert your briegel recipes to a simple |
236 |
ebuild script and run them side by side? It's actually such an easy |
237 |
thing to do, to build a new stage2/3/4 that it's a shame folks don't do |
238 |
it more! |
239 |
|
240 |
Good luck |
241 |
|
242 |
Ed W |