Gentoo Archives: gentoo-alt

From: Fabian Groffen <grobian@g.o>
To: gentoo-alt@l.g.o
Subject: Re: [gentoo-alt] portage prefix chaining support
Date: Fri, 03 Apr 2009 10:01:25
Message-Id: 20090403100107.GC1861@gentoo.org
In Reply to: Re: [gentoo-alt] portage prefix chaining support by Markus Duft
On 03-04-2009 09:04:39 +0200, Markus Duft wrote:
> On Thu, 2009-04-02 at 22:22 +0200, Fabian Groffen wrote: > > I'm a bit concerned about the design of things here. > > > > Initially, we had "cross-prefix", where Portage's EPREFIX could be > > changed at runtime. This means Portage could "manage" another prefix, > > ideally meant to clone -- bootstrap using an existing Prefix from a > > location where you don't want to have it (read-only CD?). > > I don't know how far this actually got, but I know Portage does > > cross-prefix some things if you have EPREFIX set. > > Hm - no. What you describe works out of the box i think. the portage in > prefix-launcher does all this. Cross-prefix added the capability to > (while managing EPREFIX) merge into BPREFIX. This works only for DEPEND.
"out of the box" still means that it was implemented in Prefix Portage, and I "did" it based on a patch by haubi. I don't understand the statement about merging into BPREFIX. You can set EPREFIX in your environment, and portage will then merge to it, instead of its configured prefix.
> prefix-chaining does nearly the same as cross-prefix, and the only user > visible difference is, that portage can not merge DEPENDs into BPREFIX. > that functionality made the portage patch for it a few times bigger. > also haubi seems to feel that merging to BPREFIX is somewhat wrong, so i > think prefix-chaining is the "better" cross-prefix.
It seems to me we should align this with the cross-compile efforts that IMO deal with the same problem.
> Let me bring an example: if xx DEPENDed on yy and RDEPENDed on zz, > emerge xx would result in:
xx: DEPEND=yy RDEPEND=zz
> normal: > xx, yy, zz merged in EPREFIX, BPREFIX may be used for some > executables, since it is in PATH (set by portage via > DEFAULT_PATH).
BPREFIX=EPREFIX merge yy to BPREFIX, merge zz to EPREFIX, merge xx to EPREFIX Portage's path is DEFAULT_PATH:@PORTAGE_BIN_PATH@/ebuild-helpers:ROOTPATH where DEFAULT_PATH comes from BPREFIX/etc/make.globals, and the bin-path is set during configure (hence BPREFIX). (Note: etc/make.conf comes from EPREFIX/etc/make.conf)
> cross-prefix: > xx and zz merged into EPREFIX, yy merged into BPREFIX > executables may be taken from either EPREFIX or BPREFIX, > since both are set. there is no possibility to install > portage in a "child" EPREFIX, since this would make it > forget about what was the BPREFIX (thats where i want to > take DEPENDs from).
BPREFIX!=EPREFIX merge yy to BPREFIX, merge zz to EPREFIX, merge xx to EPREFIX Since the path doesn't contain any references to EPREFIX (see above), only BPREFIX and host binaries can be found by ebuilds, this is because they are building tools. Library searches, however should be for EPREFIX (only). This is the area of gcc-config and binutils-config wrappers. It quickly gets hairy here.
> prefix-chaining: > xx merger in EPREFIX. yy and zz depends on settings in > make.conf if DEPEND and RDEPEND would both be set to be > chained, both would be taken from a chained prefix > (attention: not BPREFIX, since i'm able to have a local > portage, which still knows about "parent" prefixes). If > they are not installed in any of the (possibly multiple) > parent prefixes, portage tries to install them locally. > If one of them is masked for the current ARCH/profile, > portage will refuse to install the package. (as opposed > to cross-prefix, where portage (for DEPENDs only) would > allow to merge the package into BPREFIX if it is unmasked > there.
So, even though there is an RDEPEND, Portage satisfies it from the host (can be Prefix or non-Prefix[1]) to install xx. My observation here is that you enter an area which we[2] have always avoided to enter. The host can uninstall zz, or upgrade it to a possibly incompatible version. Conceptually, this seems to be a mess to me, and it can only be done with a big fat disclaimer, for it is a hacky solution that shouldn't be advertised to anyone that is not fully aware of what's going on. Question: is such host-aware Portage an extension or an entirely different project?
> > Now your chaining patches. > > > > Apparently you take the total opposite direction now, where you first > > have a Portage in EPREFIX (how you got it is questionable), and within > > this Portage you expect it to take stuff it works with (bash, sed) from > > a different location, let's call it CPREFIX. >
[ removing all sub-clauses here ]
> hm. you're partially right here. _if_ i install a portage instance in > EPREFIX, then i'm taking things > from any parent prefix. but i think this is ok, because at least one of > the parent prefixes _has_ to have > all this, since it has to be a fully fledged prefix.
That is not guaranteed, but it also does not convince me you have to "pull" (chain) instead of "push" (cross).
> (which i don't need to - especially when for example merging > into a "cross" prefix (interix -> winnt comes to my mind again ;)), > where i want only things compiled for $CHOST)
Even though $CHOST and $CTARGET are the same (or compatible), not providing a library that is necessary results in an object containing some RUNPATHs, but eventually a link outside EPREFIX. Nothing wrong per-se, but binpkgs and stuff will no longer work correctly.
> (which are guaranteed to be in path)
> (or maybe gentoo linux at EROOT=/ at some point.)
That is a nice case, but again, does it need a pull instead of a push?
> The only point where i had to change > things regarding this was the bash/mv issue introduced lately. i solved > this via the ebuild where i symlink bash and mv to the currently > available bash/mv - if you bootstrap right, this _has_ to be the bash > and mv from the local prefix - otherwise an emerge -e world was missing. > if it is a chained prefix, the tools from any parent are used - which i > think is ok.
The wrappers in itself seem unnecessary to me, I guess configure should just record the path to the found binaries and hardcode them in portage.const_autotool. I will code that.
> as i said above, cross-prefix was exactly the same, except that portage > could merge into BPREFIX (and that it was impossible to merge portage > into EPREFIX without breaking it (because this portage would not have > known about the "parent" anymore)). prefix-chaining simply works the > other way round - each EPREFIX knows it's parents (always, regarless of > localness of portage), as opposed to cross-prefix wher you had to tell > portage "this is your child, merge there". as for paths beeing fixed to > EPREFIX, both are the same.
The --with-default-path argument is there for a reason, bootstrap-prefix.sh actually uses it to keep the bootstrapped "prefix" available to the just installed portage.
> > I don't want to kill all of your work, but I do want to make clear to > > you why I am so hesitant for all of your changes. Conceptually, I > > believe cross-prefix was sound and clear. Now this prefix-chaining is > > like a whole new world that seemingly has some dirty concepts that I > > find hard to accept. > > hehe. haubi and me spent a reasonable amout of time discussing all this, > and we're pretty sure that this is a very good (if not the best) > approach to solving what we're trying to do (and what we require, of > course). the cross-prefix thing was a _really_ big and nearly > unmaintainable patch, since it chained the way a ROOT worked - and hell > yes, root is a word that is used often in portage code ;). > prefix-chaining is much slimer, and it is nearly the same.
But it requires some still to me unacceptable changes, that I also am sure gentoo-x86 doesn't like. Since we're on the edge of merging a lot of work into gentoo-x86, I'm not to happy with getting another flood of controversial changes to ebuilds. I guess it's mainly a missing design spec, and also a bit of sloppyness that is not appreciated. Take http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/dev-libs/boost/boost-1.35.0-r2.ebuild?rev=40636 and http://overlays.gentoo.org/proj/alt/changeset/40636, as you can see, for your purpose you conditionalise python support based on the existence of python. However, python is depended on based on x86-winnt. So this doesn't really feel consistent to me, and is a good example of why I feel pretty much like this is not the way like we want to solve these kinds of things. Just let alone that we change the way boost is built this way.
> actually, one can do the same with chaining as with cross-prefix, by > giving prefix-chain-setup the --minimal flag, which makes it omit > portage and gcc-config, and just merge baselayout and > prefix-chain-utils. > > sorry - i'm not too good in explaining - i'm just jumping around too > much. maybe haubi can further clearyfy all this a little? thanks...
[1] yes, some people really would like to reuse stuff from the host OS. [2] Mac OS X members of the Gentoo Prefix team after the Gentoo for Mac OS X disaster. -- Fabian Groffen Gentoo on a different level