1 |
While some of you may have heard of it, we -- the Prefix team -- have |
2 |
the impression that for most developers the Gentoo Prefix project is in |
3 |
general an unknown, hidden and vague project, that primarily generates a |
4 |
lot of commits. Therefore, we have decided to try and explain what |
5 |
Gentoo Prefix is, and why we Prefix devs are so passionately about it. |
6 |
|
7 |
Let us start by pointing out that Gentoo Prefix is all but a "new" |
8 |
project. While the archeological signs of the project only go back to |
9 |
2006[1], the project has existed before that in the minds of people no |
10 |
longer with us any more. |
11 |
|
12 |
The Gentoo Prefix project aims to bring the Gentoo Portage Tree to |
13 |
non-Gentoo and/or non-Linux systems, through the following two |
14 |
characteristics: |
15 |
1) a free to choose target directory offset (prefix), in which |
16 |
packages are installed, and |
17 |
2) the ability to install and operate without adminstrative privileges |
18 |
|
19 |
To understand why these two characteristics were chosen, it is best to |
20 |
get into the mindset of many Gentoo Prefix users (and devs). There are |
21 |
plenty of reasons, of which a few described in [2]. In short, the users |
22 |
are looking for convenience on their platform of choice, or platform |
23 |
they are doomed to use. |
24 |
|
25 |
Some people just have to work with Windows. Others just like Mac OS X |
26 |
over any GNOME or KDE. Some work with Solaris, others with AIX, or even |
27 |
their Atari. All of these systems have their merits and demerits. |
28 |
(Gentoo) Linux users are used to a very rich and convenient userland |
29 |
consisting of many GNU tools, such as sed, awk, grep, tar, etc., but |
30 |
also development tools such as compilers, autotools, etc. While these |
31 |
tools are absent, simply outdated or not as flexible as on Gentoo, users |
32 |
of said systems are in need of the flexible tool suite as delivered by |
33 |
Gentoo. An obvious solution is to get Portage running and to install |
34 |
the tools on the system in use. However, due to requirements, |
35 |
privileges, or an insane love relation, the original system may not be |
36 |
harmed by replacing or modifying existing software. Yes, in some cases, |
37 |
such modifications are even impossible by lack of privileges: no root |
38 |
access! |
39 |
|
40 |
In a really compact form, we can say that for Gentoo Prefix users: |
41 |
- an offset installation is necessary not to break the host OS software |
42 |
- sometimes root privileges are unavailable |
43 |
- unprivileged installations add an extra level of protection/safety |
44 |
- `rm -Rf <prefix-offset-location>` removes the entire Prefix and all |
45 |
traces of it |
46 |
|
47 |
Gentoo Prefix delivers unprivileged offset installations, which we call |
48 |
a "prefix". The prefix here is the offset used for the entire |
49 |
installation, such as /home/sally/gentoo. A Gentoo Prefix system is |
50 |
bootstrapped into such prefix, by following a number of instructions, |
51 |
e.g.[3]. |
52 |
|
53 |
Using this prefix, the file system layout chosen by Prefix Portage is |
54 |
exactly the same as for a normal Gentoo Linux system, but shifted into |
55 |
the prefix. This would for instance result in |
56 |
/home/sally/gentoo/usr/bin for the previously mentioned example. In |
57 |
principle, it serves no use to have programs installed in bin, usr/bin, |
58 |
sbin, usr/sbin, etc. under the prefix, and it would make more sense if |
59 |
they would be installed all in one place. However, following the Gentoo |
60 |
Linux layout enables direct backwards compatability when the chosen |
61 |
prefix is the empty string, and requires no extra patching or changes. |
62 |
|
63 |
With this in mind, we would like to stress that it is not the desire of |
64 |
the current Gentoo Prefix project to go beyond the ability to install in |
65 |
an unprivileged install-time fixed offset. We consider the current |
66 |
approach as a milestone that delivers offset installations. Pilots |
67 |
within our own team have shown that on top of this milestone, further |
68 |
steps can be made, such as variable offset locations and close |
69 |
cooperation with e.g. a Gentoo Linux host system. However, we do not |
70 |
consider these works in progress as part of the *current* Gentoo Prefix |
71 |
project. |
72 |
|
73 |
Unfortunately, support for installing packages into a prefix, does not |
74 |
come for free. Most ebuilds need small changes, some ebuilds need |
75 |
extensive patching to get programs to install and run correctly with |
76 |
regard to the (offset) prefix. To get this all running, we set up our |
77 |
own tree, which contains all our "converted" ebuilds, extra profiles and |
78 |
modified eclasses[4]. At some point our tree became pretty big[1], and |
79 |
it turned out our enthusiasm (and that of our users) brought |
80 |
overlays.g.o down to its knees. At that point, our hobby got way out of |
81 |
hand, and to resolve the issue we had to switch to rsync, resulting in |
82 |
our own generated tree with metadata. As side-effect this greatly sped |
83 |
up the user experience[5], contributing to getting our hobby even more |
84 |
out of hand. We, as Prefix team, feel this puts our project in a bit |
85 |
weird position, where we resemble closely to an in-house fork of the |
86 |
Gentoo project itself. |
87 |
|
88 |
It is our desire and intention to move away from this situation, which |
89 |
not only consumes an awful lot of resources for maintaining the fork, |
90 |
but also keeps a lot of work outside of the mainstream Gentoo |
91 |
infrastructure. |
92 |
|
93 |
As Prefix team, we feel that the current shape of the Gentoo Prefix |
94 |
implementation is mature. That is, it has been proven to be useful for |
95 |
a number of users/scenarios, and it has been able to work for a |
96 |
substantial number of different systems, without having to change the |
97 |
core part. |
98 |
|
99 |
Therefore, we plan to focus on merging back the many patches and |
100 |
extensions from our Prefix overlay into the Gentoo mainline tree. For |
101 |
us, this roadmap looks as follows: |
102 |
1) get the prefix USE-flag masked in profiles/base |
103 |
We use a special USE-flag 'prefix' to conditionalise actions in |
104 |
ebuilds and eclasses that really depend on being in an offset |
105 |
installation or not. Obviously, this flag should be disabled in |
106 |
normal profiles. |
107 |
2) move the prefix profiles to the mainline tree |
108 |
The Prefix project maintains a profiles/prefix directory with Gentoo |
109 |
Prefix specific profiles. They inherit from base, and in the Linux |
110 |
case even from the default/linux profiles. |
111 |
3) add prefix-only ebuilds to the mainline tree |
112 |
Gentoo Prefix has different systems, which sometimes need different |
113 |
packages. Specialised ebuilds now live in the Prefix tree only. |
114 |
4) file bugs for eclass changes to their maintainers |
115 |
Like ebuilds, eclasses need changes to work properly using an offset |
116 |
installation. These changes are usually related to dealing with the |
117 |
offset prefix, but sometimes also add specialised support for the |
118 |
Prefix architectures. |
119 |
5) bring back trivial patches |
120 |
Different systems bring different compile or installation problems. |
121 |
These patches can be trivial, and hence non-intrusive. We like to |
122 |
move those patches to the Gentoo mainline tree. |
123 |
6) solve complicated ebuild changes |
124 |
For a number of ebuilds, mostly for the toolchain, a set of extensive |
125 |
patches are applied in the Prefix tree. While our changes are mostly |
126 |
designed to be backwards compatible, we prefer to get them reviewed |
127 |
by the respective maintainers and get them merged with their |
128 |
blessings. |
129 |
7) merging prefix branch of Portage into trunk |
130 |
Prefix Portage currently lives in its own branch of the Portage |
131 |
sources. With an empty offset string, Prefix Portage is equal in |
132 |
behaviour to trunk sources. We would like to merge the Prefix branch |
133 |
with trunk and have a release, instead of keeping the branch and |
134 |
adding a Prefix Portage ebuild. |
135 |
8) add prefix bits to ebuilds |
136 |
Prefix involves some variables which need to be used in ebuilds |
137 |
to make them offset-aware. Often this is trivial, but requires |
138 |
understanding on what is exactly going on. We plan to provide |
139 |
information about this. At the same time we do this, we want to move |
140 |
over our Prefix arches keywords, such that we can start using the |
141 |
Gentoo mainline tree. |
142 |
|
143 |
We hope this informative email sheds a light on the activities of the |
144 |
Gentoo Prefix team, and the actions we aim to take in the near future. |
145 |
We are open for any questions or comments. |
146 |
|
147 |
|
148 |
[1] http://stats.prefix.freens.org/keywords-packages.png |
149 |
[2] http://www.gentoo.org/proj/en/gentoo-alt/prefix/usecases.xml |
150 |
[3] http://www.gentoo.org/proj/en/gentoo-alt/prefix/bootstrap-solaris.xml |
151 |
[4] http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay |
152 |
[5] http://stats.prefix.freens.org/rsync-usage.png |
153 |
|
154 |
|
155 |
-- |
156 |
Fabian Groffen |
157 |
Gentoo on a different level |