Gentoo Archives: gentoo-alt

From: Fabian Groffen <grobian@g.o>
To: gentoo-alt@l.g.o
Subject: Re: [gentoo-alt] portage prefix chaining support
Date: Fri, 03 Apr 2009 10:01:25
Message-Id: 20090403100107.GC1861@gentoo.org
In Reply to: Re: [gentoo-alt] portage prefix chaining support by Markus Duft
1 On 03-04-2009 09:04:39 +0200, Markus Duft wrote:
2 > On Thu, 2009-04-02 at 22:22 +0200, Fabian Groffen wrote:
3 > > I'm a bit concerned about the design of things here.
4 > >
5 > > Initially, we had "cross-prefix", where Portage's EPREFIX could be
6 > > changed at runtime. This means Portage could "manage" another prefix,
7 > > ideally meant to clone -- bootstrap using an existing Prefix from a
8 > > location where you don't want to have it (read-only CD?).
9 > > I don't know how far this actually got, but I know Portage does
10 > > cross-prefix some things if you have EPREFIX set.
11 >
12 > Hm - no. What you describe works out of the box i think. the portage in
13 > prefix-launcher does all this. Cross-prefix added the capability to
14 > (while managing EPREFIX) merge into BPREFIX. This works only for DEPEND.
15
16 "out of the box" still means that it was implemented in Prefix Portage,
17 and I "did" it based on a patch by haubi. I don't understand the
18 statement about merging into BPREFIX. You can set EPREFIX in your
19 environment, and portage will then merge to it, instead of its
20 configured prefix.
21
22 > prefix-chaining does nearly the same as cross-prefix, and the only user
23 > visible difference is, that portage can not merge DEPENDs into BPREFIX.
24 > that functionality made the portage patch for it a few times bigger.
25 > also haubi seems to feel that merging to BPREFIX is somewhat wrong, so i
26 > think prefix-chaining is the "better" cross-prefix.
27
28 It seems to me we should align this with the cross-compile efforts that
29 IMO deal with the same problem.
30
31 > Let me bring an example: if xx DEPENDed on yy and RDEPENDed on zz,
32 > emerge xx would result in:
33
34 xx:
35 DEPEND=yy
36 RDEPEND=zz
37
38 > normal:
39 > xx, yy, zz merged in EPREFIX, BPREFIX may be used for some
40 > executables, since it is in PATH (set by portage via
41 > DEFAULT_PATH).
42
43 BPREFIX=EPREFIX
44 merge yy to BPREFIX, merge zz to EPREFIX, merge xx to EPREFIX
45
46 Portage's path is
47 DEFAULT_PATH:@PORTAGE_BIN_PATH@/ebuild-helpers:ROOTPATH
48 where DEFAULT_PATH comes from BPREFIX/etc/make.globals, and the bin-path
49 is set during configure (hence BPREFIX).
50 (Note: etc/make.conf comes from EPREFIX/etc/make.conf)
51
52 > cross-prefix:
53 > xx and zz merged into EPREFIX, yy merged into BPREFIX
54 > executables may be taken from either EPREFIX or BPREFIX,
55 > since both are set. there is no possibility to install
56 > portage in a "child" EPREFIX, since this would make it
57 > forget about what was the BPREFIX (thats where i want to
58 > take DEPENDs from).
59
60 BPREFIX!=EPREFIX
61 merge yy to BPREFIX,
62 merge zz to EPREFIX, merge xx to EPREFIX
63
64 Since the path doesn't contain any references to EPREFIX (see above),
65 only BPREFIX and host binaries can be found by ebuilds, this is because
66 they are building tools. Library searches, however should be for
67 EPREFIX (only). This is the area of gcc-config and binutils-config
68 wrappers. It quickly gets hairy here.
69
70 > prefix-chaining:
71 > xx merger in EPREFIX. yy and zz depends on settings in
72 > make.conf if DEPEND and RDEPEND would both be set to be
73 > chained, both would be taken from a chained prefix
74 > (attention: not BPREFIX, since i'm able to have a local
75 > portage, which still knows about "parent" prefixes). If
76 > they are not installed in any of the (possibly multiple)
77 > parent prefixes, portage tries to install them locally.
78 > If one of them is masked for the current ARCH/profile,
79 > portage will refuse to install the package. (as opposed
80 > to cross-prefix, where portage (for DEPENDs only) would
81 > allow to merge the package into BPREFIX if it is unmasked
82 > there.
83
84 So, even though there is an RDEPEND, Portage satisfies it from the host
85 (can be Prefix or non-Prefix[1]) to install xx. My observation here is
86 that you enter an area which we[2] have always avoided to enter. The
87 host can uninstall zz, or upgrade it to a possibly incompatible version.
88 Conceptually, this seems to be a mess to me, and it can only be done
89 with a big fat disclaimer, for it is a hacky solution that shouldn't be
90 advertised to anyone that is not fully aware of what's going on.
91
92 Question: is such host-aware Portage an extension or an entirely
93 different project?
94
95 > > Now your chaining patches.
96 > >
97 > > Apparently you take the total opposite direction now, where you first
98 > > have a Portage in EPREFIX (how you got it is questionable), and within
99 > > this Portage you expect it to take stuff it works with (bash, sed) from
100 > > a different location, let's call it CPREFIX.
101 >
102 [ removing all sub-clauses here ]
103 > hm. you're partially right here. _if_ i install a portage instance in
104 > EPREFIX, then i'm taking things
105 > from any parent prefix. but i think this is ok, because at least one of
106 > the parent prefixes _has_ to have
107 > all this, since it has to be a fully fledged prefix.
108
109 That is not guaranteed, but it also does not convince me you have to
110 "pull" (chain) instead of "push" (cross).
111
112 > (which i don't need to - especially when for example merging
113 > into a "cross" prefix (interix -> winnt comes to my mind again ;)),
114 > where i want only things compiled for $CHOST)
115
116 Even though $CHOST and $CTARGET are the same (or compatible), not
117 providing a library that is necessary results in an object containing
118 some RUNPATHs, but eventually a link outside EPREFIX. Nothing wrong
119 per-se, but binpkgs and stuff will no longer work correctly.
120
121 > (which are guaranteed to be in path)
122
123 > (or maybe gentoo linux at EROOT=/ at some point.)
124
125 That is a nice case, but again, does it need a pull instead of a push?
126
127 > The only point where i had to change
128 > things regarding this was the bash/mv issue introduced lately. i solved
129 > this via the ebuild where i symlink bash and mv to the currently
130 > available bash/mv - if you bootstrap right, this _has_ to be the bash
131 > and mv from the local prefix - otherwise an emerge -e world was missing.
132 > if it is a chained prefix, the tools from any parent are used - which i
133 > think is ok.
134
135 The wrappers in itself seem unnecessary to me, I guess configure should
136 just record the path to the found binaries and hardcode them in
137 portage.const_autotool. I will code that.
138
139 > as i said above, cross-prefix was exactly the same, except that portage
140 > could merge into BPREFIX (and that it was impossible to merge portage
141 > into EPREFIX without breaking it (because this portage would not have
142 > known about the "parent" anymore)). prefix-chaining simply works the
143 > other way round - each EPREFIX knows it's parents (always, regarless of
144 > localness of portage), as opposed to cross-prefix wher you had to tell
145 > portage "this is your child, merge there". as for paths beeing fixed to
146 > EPREFIX, both are the same.
147
148 The --with-default-path argument is there for a reason,
149 bootstrap-prefix.sh actually uses it to keep the bootstrapped "prefix"
150 available to the just installed portage.
151
152 > > I don't want to kill all of your work, but I do want to make clear to
153 > > you why I am so hesitant for all of your changes. Conceptually, I
154 > > believe cross-prefix was sound and clear. Now this prefix-chaining is
155 > > like a whole new world that seemingly has some dirty concepts that I
156 > > find hard to accept.
157 >
158 > hehe. haubi and me spent a reasonable amout of time discussing all this,
159 > and we're pretty sure that this is a very good (if not the best)
160 > approach to solving what we're trying to do (and what we require, of
161 > course). the cross-prefix thing was a _really_ big and nearly
162 > unmaintainable patch, since it chained the way a ROOT worked - and hell
163 > yes, root is a word that is used often in portage code ;).
164 > prefix-chaining is much slimer, and it is nearly the same.
165
166 But it requires some still to me unacceptable changes, that I also am
167 sure gentoo-x86 doesn't like. Since we're on the edge of merging a lot
168 of work into gentoo-x86, I'm not to happy with getting another flood of
169 controversial changes to ebuilds. I guess it's mainly a missing design
170 spec, and also a bit of sloppyness that is not appreciated.
171 Take
172 http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/dev-libs/boost/boost-1.35.0-r2.ebuild?rev=40636
173 and http://overlays.gentoo.org/proj/alt/changeset/40636, as you can see,
174 for your purpose you conditionalise python support based on the
175 existence of python. However, python is depended on based on x86-winnt.
176 So this doesn't really feel consistent to me, and is a good example of
177 why I feel pretty much like this is not the way like we want to solve
178 these kinds of things. Just let alone that we change the way boost is
179 built this way.
180
181 > actually, one can do the same with chaining as with cross-prefix, by
182 > giving prefix-chain-setup the --minimal flag, which makes it omit
183 > portage and gcc-config, and just merge baselayout and
184 > prefix-chain-utils.
185 >
186 > sorry - i'm not too good in explaining - i'm just jumping around too
187 > much. maybe haubi can further clearyfy all this a little? thanks...
188
189
190 [1] yes, some people really would like to reuse stuff from the host OS.
191 [2] Mac OS X members of the Gentoo Prefix team after the Gentoo for Mac
192 OS X disaster.
193
194 --
195 Fabian Groffen
196 Gentoo on a different level