Gentoo Archives: gentoo-dev

From: Fabian Groffen <grobian@g.o>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] An Introduction to Gentoo Prefix
Date: Tue, 05 May 2009 18:18:23
1 While some of you may have heard of it, we -- the Prefix team -- have
2 the impression that for most developers the Gentoo Prefix project is in
3 general an unknown, hidden and vague project, that primarily generates a
4 lot of commits. Therefore, we have decided to try and explain what
5 Gentoo Prefix is, and why we Prefix devs are so passionately about it.
7 Let us start by pointing out that Gentoo Prefix is all but a "new"
8 project. While the archeological signs of the project only go back to
9 2006[1], the project has existed before that in the minds of people no
10 longer with us any more.
12 The Gentoo Prefix project aims to bring the Gentoo Portage Tree to
13 non-Gentoo and/or non-Linux systems, through the following two
14 characteristics:
15 1) a free to choose target directory offset (prefix), in which
16 packages are installed, and
17 2) the ability to install and operate without adminstrative privileges
19 To understand why these two characteristics were chosen, it is best to
20 get into the mindset of many Gentoo Prefix users (and devs). There are
21 plenty of reasons, of which a few described in [2]. In short, the users
22 are looking for convenience on their platform of choice, or platform
23 they are doomed to use.
25 Some people just have to work with Windows. Others just like Mac OS X
26 over any GNOME or KDE. Some work with Solaris, others with AIX, or even
27 their Atari. All of these systems have their merits and demerits.
28 (Gentoo) Linux users are used to a very rich and convenient userland
29 consisting of many GNU tools, such as sed, awk, grep, tar, etc., but
30 also development tools such as compilers, autotools, etc. While these
31 tools are absent, simply outdated or not as flexible as on Gentoo, users
32 of said systems are in need of the flexible tool suite as delivered by
33 Gentoo. An obvious solution is to get Portage running and to install
34 the tools on the system in use. However, due to requirements,
35 privileges, or an insane love relation, the original system may not be
36 harmed by replacing or modifying existing software. Yes, in some cases,
37 such modifications are even impossible by lack of privileges: no root
38 access!
40 In a really compact form, we can say that for Gentoo Prefix users:
41 - an offset installation is necessary not to break the host OS software
42 - sometimes root privileges are unavailable
43 - unprivileged installations add an extra level of protection/safety
44 - `rm -Rf <prefix-offset-location>` removes the entire Prefix and all
45 traces of it
47 Gentoo Prefix delivers unprivileged offset installations, which we call
48 a "prefix". The prefix here is the offset used for the entire
49 installation, such as /home/sally/gentoo. A Gentoo Prefix system is
50 bootstrapped into such prefix, by following a number of instructions,
51 e.g.[3].
53 Using this prefix, the file system layout chosen by Prefix Portage is
54 exactly the same as for a normal Gentoo Linux system, but shifted into
55 the prefix. This would for instance result in
56 /home/sally/gentoo/usr/bin for the previously mentioned example. In
57 principle, it serves no use to have programs installed in bin, usr/bin,
58 sbin, usr/sbin, etc. under the prefix, and it would make more sense if
59 they would be installed all in one place. However, following the Gentoo
60 Linux layout enables direct backwards compatability when the chosen
61 prefix is the empty string, and requires no extra patching or changes.
63 With this in mind, we would like to stress that it is not the desire of
64 the current Gentoo Prefix project to go beyond the ability to install in
65 an unprivileged install-time fixed offset. We consider the current
66 approach as a milestone that delivers offset installations. Pilots
67 within our own team have shown that on top of this milestone, further
68 steps can be made, such as variable offset locations and close
69 cooperation with e.g. a Gentoo Linux host system. However, we do not
70 consider these works in progress as part of the *current* Gentoo Prefix
71 project.
73 Unfortunately, support for installing packages into a prefix, does not
74 come for free. Most ebuilds need small changes, some ebuilds need
75 extensive patching to get programs to install and run correctly with
76 regard to the (offset) prefix. To get this all running, we set up our
77 own tree, which contains all our "converted" ebuilds, extra profiles and
78 modified eclasses[4]. At some point our tree became pretty big[1], and
79 it turned out our enthusiasm (and that of our users) brought
80 overlays.g.o down to its knees. At that point, our hobby got way out of
81 hand, and to resolve the issue we had to switch to rsync, resulting in
82 our own generated tree with metadata. As side-effect this greatly sped
83 up the user experience[5], contributing to getting our hobby even more
84 out of hand. We, as Prefix team, feel this puts our project in a bit
85 weird position, where we resemble closely to an in-house fork of the
86 Gentoo project itself.
88 It is our desire and intention to move away from this situation, which
89 not only consumes an awful lot of resources for maintaining the fork,
90 but also keeps a lot of work outside of the mainstream Gentoo
91 infrastructure.
93 As Prefix team, we feel that the current shape of the Gentoo Prefix
94 implementation is mature. That is, it has been proven to be useful for
95 a number of users/scenarios, and it has been able to work for a
96 substantial number of different systems, without having to change the
97 core part.
99 Therefore, we plan to focus on merging back the many patches and
100 extensions from our Prefix overlay into the Gentoo mainline tree. For
101 us, this roadmap looks as follows:
102 1) get the prefix USE-flag masked in profiles/base
103 We use a special USE-flag 'prefix' to conditionalise actions in
104 ebuilds and eclasses that really depend on being in an offset
105 installation or not. Obviously, this flag should be disabled in
106 normal profiles.
107 2) move the prefix profiles to the mainline tree
108 The Prefix project maintains a profiles/prefix directory with Gentoo
109 Prefix specific profiles. They inherit from base, and in the Linux
110 case even from the default/linux profiles.
111 3) add prefix-only ebuilds to the mainline tree
112 Gentoo Prefix has different systems, which sometimes need different
113 packages. Specialised ebuilds now live in the Prefix tree only.
114 4) file bugs for eclass changes to their maintainers
115 Like ebuilds, eclasses need changes to work properly using an offset
116 installation. These changes are usually related to dealing with the
117 offset prefix, but sometimes also add specialised support for the
118 Prefix architectures.
119 5) bring back trivial patches
120 Different systems bring different compile or installation problems.
121 These patches can be trivial, and hence non-intrusive. We like to
122 move those patches to the Gentoo mainline tree.
123 6) solve complicated ebuild changes
124 For a number of ebuilds, mostly for the toolchain, a set of extensive
125 patches are applied in the Prefix tree. While our changes are mostly
126 designed to be backwards compatible, we prefer to get them reviewed
127 by the respective maintainers and get them merged with their
128 blessings.
129 7) merging prefix branch of Portage into trunk
130 Prefix Portage currently lives in its own branch of the Portage
131 sources. With an empty offset string, Prefix Portage is equal in
132 behaviour to trunk sources. We would like to merge the Prefix branch
133 with trunk and have a release, instead of keeping the branch and
134 adding a Prefix Portage ebuild.
135 8) add prefix bits to ebuilds
136 Prefix involves some variables which need to be used in ebuilds
137 to make them offset-aware. Often this is trivial, but requires
138 understanding on what is exactly going on. We plan to provide
139 information about this. At the same time we do this, we want to move
140 over our Prefix arches keywords, such that we can start using the
141 Gentoo mainline tree.
143 We hope this informative email sheds a light on the activities of the
144 Gentoo Prefix team, and the actions we aim to take in the near future.
145 We are open for any questions or comments.
148 [1]
149 [2]
150 [3]
151 [4]
152 [5]
155 --
156 Fabian Groffen
157 Gentoo on a different level


Subject Author
[gentoo-dev] Re: An Introduction to Gentoo Prefix Christian Faulhammer <fauli@g.o>
Re: [gentoo-dev] An Introduction to Gentoo Prefix Markos Chandras <hwoarang@g.o>
Re: [gentoo-dev] An Introduction to Gentoo Prefix Mounir Lamouri <volkmar@g.o>