Re: [gentoo-dev] BUILDROOT concept - what about multilib?

[Travis Tilley]

On Friday 23 April 2004 11:04 am, Paul Smith wrote:
> What I want to be able to do is this: have one (or more) Portage
> installations, or "images", on a system.  An "image" is a complete
> Portage system, but "virtualized" by having it rooted somewhere besides
> /.  Suppose you had a directory like /opt/portage, and under there you
> had p1, p2, p3, etc. which were Portage images.  Then underneath each
> one of those would be, for example, an etc/portage directory, and a
> var/log/emerge.log, and a var/db/pkg, etc. which contained Portage
> control files/databases for only that image.

*ears perk up*
keep me posted on your success here. what i would like to do with portage
varies greatly from what you seem to want, but it depends on the exact same
functionality, with slightly different limitations.

what /i/ would love to see in portage is true multilib compatibility for
 archs with a bi-arch toolchain, with seperate dependency trees for each
 bitdepth that is supported. in the case of amd64, that would mean seperate
 dependency trees for 64bit and 32bit.

right now, on amd64, we are as non-compliant as it can get in regards to
standards and it makes quite a few things difficult sometimes... but a lot of
things want to install to lib and not lib64, and nobody really wants to
bother rewriting econf as well as any ebuild that doesn't use it when we dont
even have support for the multiple dependency trees that would justify such
work.

another oddity brought on by the bi-arch toolchain problem that would
complicate things even more is that unlike your cross-compiler case, in a
multilib situation you would need cross-tree dependencies and everything from
both trees will be installed to the same ROOT, using the same BUILDROOT. so
this part of your work probably wont help the multilib case (dont mind me,
i'm not a python coder and i'm already thinking about how to hack whatever
you end up working on to do what i want it to ^_^ ). we would need to have
only one gcc, glibc, etc... and it would have to be 64bit on amd64. on amd64,
the 64bit toolchain would need to be used to build anything in the 32bit dep
tree, only with the forced addition of -m32 everywhere.

> Now when you want to install a new package, before you can run emerge
> you need to choose a "build", or host, image and a "destination", or
> target, image.  We'll say that the destination is selected by the value
> of ROOT, and the build image is selected by the value of BUILDROOT.

i can also see any work you do here helping out in the multilib case, if it's
taken into account. however, instead of ROOT and BUILDROOT there would have
to be a way to handle ROOTARCH and BUILDROOTARCH, or something similar... in
the multilib case, at least for amd64, we would want 32bit libraries to end
up in /lib, and 32bit binaries to have the suffix "32". and the binary
package itself would have to decide it's own arch... there would be no other
way of knowing what the package contains without unpacking it. and then we
wouldn't know if the package doesn't contain anything arch-specific we could
test... hmm.

> Obviously the simplest case is where both ROOT and BUILDROOT are "/"
> (which would be the default value).  This represents the standard
> Portage usage.

for the multilib case, this is anything but simple :'(

> The idea is you could create new Portage images quite independently from
> your real system.  In particular, the new Portage image could be for a
> completely different architecture: cross-compilers could be installed in
> the "build" image, then used to build and install into the
> "destination".

i'm much more interested in the ability to use the amd64 native 64bit
toolchain to build 64bit, 32bit, and bi-arch installs without use of a
cross-compiler or having to drop into a 32bit chroot. :)

> The above is not such a big deal; it could play relatively nicely I
> think with the current Portage.  We just need to change references to
> fully-qualified paths with variable-qualified paths based on the
> BUILDROOT value.

for the multilib case we would need additional variables to decide where
things go, and such. :/

> However, the problem of the current handling of DEPEND vs. RDEPEND is
> not so simple to resolve.  It looks like DEPEND and RDEPEND are targeted
> at a very different scenario than the one I outline above.  If ROOT is
> set, it appears that Portage looks for DEPEND packages on the "real"
> system and RDEPEND packages in the ROOT image.  Also, all the Portage
> database info, logs, etc. is kept in the "real" root, not the ROOT root.
> I guess I'm not quite clear on what scenario this setup actually
> supports?  Can anyone give me a quick use-case?  It almost seems like
> you're trying to support creating non-Portage-based images from a
> Portage-based image... that's not something I'm really that interested
> in.

multilib will probably also require an arch-less or arch-of-choice dependency
that specifies when a package could really care less what arch the dependency
is. scripts and interpreted languages dont give a damn what arch the shell
and interpreter are, for example. also, for a functional install, a multilib
system probably doesnt want two seperate copies of certain things installed
to ROOT... i'm pretty sure we only need one kernel logger, init, bash, etc.
i'd say most if not all of what would appear in a stage3 that isn't a library
(or glibc, which is multilib aware and will install as 32bit and 64bit if
told to).

so, also, three dependency trees... where the two bi-arch trees include deps
from a third tree containing stuff that shouldn't be installed twice? *shrug*

> In my environment the above would not be correct: both DEPEND and
> RDEPEND packages would have to appear in the ROOT image (remember that
> the BUILDROOT and ROOT could be completely different architectures!)

ditto. kinda.

> SO!  Does any of that make sense?  Is there some way we can rationalize
> the handling of DEPEND/RDEPEND?  Maybe something like if BUILDROOT is
> set then DEPEND/RDEPEND are both looked for in the ROOT?  I don't really
> like that as it seems like "special magic" that's kind of difficult to
> understand, but it is backward-compatible.

i'm thinking that if you take into consideration just cross-compiling, there
would have to be additional hacks added later on if portage ever wants to
support multilib. but if you take multilib into consideration just a little
while working on cross-compiling, we might end up getting multilib support
almost for free. and if that happens, dude i will love you for LIFE.

you have no idea how much of a PITA it is to support 32bit anything in
 gentoo. if we were multilib compliant, you could almost install packages
 from x86 gentoo on amd64 without modification. all we would need is some
 editing to the ebuild install to rename 32bit apps, perhaps rename env.d
 entries, perhaps add a script in post-install for the real name that will
 run the 32bit version... etc. similar workarounds could be provided for
 64bit installs to make sure that /all/ 64bit libs end up in a lib64
 directory...

hmm. yeah. dont mind me for thinking out loud to the list, but it would
definately make me VERY happy to see such functionality in portage. you have
no idea... even after all the work it would take to work around the current
UGLY FREAKING HACKS we use and any additional work needed to provide
compatibility/migration for our current hacked-as-fsck ugly-as-sin installs.


Travis Tilley <lv@g.o>
Gentoo/AMD64 Developer

--
gentoo-dev@g.o mailing list