Re: [gentoo-alt] Crossdev in a prefix / bug 384167

[Greg Turner]

On Wed, Jun 12, 2013 at 8:40 PM, Ruud Koolen <redlizard@×××××××××.nl> wrote:
> On Monday 10 June 2013 12:17:44 gmt@×××××.us wrote:
>> I have crossdev starting to work in a prefix, via an overlay.  So far, I
>> have x86_64-pc-linux-gnu->i686-pc-linux-gnu[-multilib] working, and I seem
>> to be making good progress towards my intermediate objective,
>> x86_64-pc-linux-gnu->x86-64-w64-mingw64[multilib].
>
> heroxbd and me are working on that too, as part of
> http://wiki.gentoo.org/wiki/Google_Summer_of_Code/2013/Ideas/Gentoo_on_Android .
> The idea so far is that crossdev should work more or less automatically once
> we manage to implement some toolchain fixes; basically, the toolchain
> packages (binutils, gcc, libc) and portage should support cross-compiling
> across different prefixes as well as across different architectures, after
> which the crossdev script should just work.

A number of places in the existing e{builds,classes} assume,
approximately (and in pseudo-pseudocode :P) that:

    ( ${CHOST} != ${C{BUILD,TARGET}} )
    ===> ( EPREFIX == "" )


In English, I mean, that at any given moment, one is permitted by the
code to be either cross-compiling or to be using prefixed-portage, but
not both.

The changes I've made so far are lowest-hanging-fruit -- they don't
really change this assumption.  Instead what they do is extend
prefix-compatibility to the crossdev script and a few key toolchain
ebuilds.  The existing logic, translates, roughly, to this:

    When I am built according to 'the "X" convention'
    (see below for a partial definition) I assume and/or
    require that EPREFIX is "", and I go through the
    configure and compile phases "pretending" that ROOT
    is "/".  Then, I ask the toolchain build scripts to make a
    cross-compiling environment which, in turn, creates its
    own sub-ROOT at /usr/${CTARGET}.  During the install
    phase, I then specify (effectively, if not literally)
    DESTDIR=${D}${ROOT}, so that the cross-sub-root ends up
    installed to as ${ROOT}usr/${CTARGET}.

As you can see this is all pretty standard.  The only "trick" is that
under no circumstances may knowledge of ROOT leak into these toolchain
e-builds during any build phases, and get mixed-up with the sub-ROOT.
IIUC we want to be able to "chroot $(portageq envvar ROOT)" and have
the right thing happen, which in this case means that within that
chroot, with CHOST unchanged, we are able to build stuff for our
cross-platform sub-ROOT by cross-emerging, which would itself be
chroot-able -- at least, I think that's right.

One thing I still haven't entirely figured out is what, exactly,
constitutes the fully correct specification of 'the "X" convention'.
The simplest and most intuitive version of this is "${CHOST} !=
${CTARGET}" but once we permit "cross-multilib-ness" to occur as well,
those tests potentially break down.  I see some deficiencies in the
current non-ROOT, non-EPREFIX case pertaining to this in gx86.

I think the changes required are actually quite easy and don't really
add much complexity; in fact they sometimes make it slightly simpler,
if you can believe it.  Here's how I'm trying to change the above to
be EPREFIX-compatible.

    When I am built according to 'the "X" convention'
    I go through the configure and compile phases
    "pretending" that ROOT is "/" and EPREFIX is
    unchanged.  Then, I ask the toolchain build scripts
    to make a cross-compiling environment which, in turn,
    creates its own sub-root at
    ${EPREFIX}/usr/${CTARGET}.  During the install
    phase, I then specify (effectively, if not literally)
    DESTDIR=${ED}${ROOT}, so the cross-sub-root ends up
    installed as ${EROOT}usr/${CTARGET}.

Bascially, there's nothing novel about the above, it just applies the
"ebuild modifications" outlined at
http://www.gentoo.org/proj/en/gentoo-alt/prefix/techdocs.xml to the
previous version.  Just as we could chroot ${ROOT} and cross-build in
the previous framework, we could now in theory chroot ${ROOT}, set
PATH="${EPREFIX}/bin:(insert standard stuff here):${PATH}", and do
whatever else constitutes activating the prefix portage, and then use
the prefixed, chrooted cross-emerge.

So returning to my initial point, this is the easy version.  The
/hard/ version pertains to the case that within the cross-emerge black
box, we want a second, possibly different EPREFIX -- we might call
this the CROSS_${CTARGET}_EPREFIX.  Put another way, I'm extending
crossdev to support a prefix on the host, but it still won't support a
prefix on the /target/.

I'm pretty sure it wouldn't be /that/ hard to actually go all the way
and support both.  I'm quite sure there are real-world applications
that actually call for that.  It's harder, though, because everywhere
-- not just in the toolchain ebuilds -- that currently assumes or
asserts that ( ${CHOST} != ${CTARGET} ) ===> ( ${EPREFIX} == "" )
would need to be fixed.  In my easy solution, we don't care about
this, since it's always true in regular non-cross portage, and it's
still always true in the cross-emerge black-box (note -- so far my
patches don't touch the cross-emerge wrapper scripts and I haven't
tested them.  I'm hoping, if they don't work right OOTB, they'll be
relatively easy to fix.  I haven't even tested with ROOT != "/",
either, and wouldn't be surprised if I still had ROOT+EPREFIX bugs as
well).

Unfortunately, also, at some point, this whole mess is going to start
bumping up against the limitations of prefix portage; namely, that it
has not yet been ported beyond the deprecated no-multilib profile.  I
am expecting this to break PREFIX -> multilib cross emerges, at least
until the prefix and non-prefix versions of gcc-config are unified.
To fix that, we could either fix prefix portage to support multilib
(better for gentoo) or give the "ESUBROOT"s (made-up fake variable for
${EROOT}/usr/${CTARGET}/) their own external ${PORTDIR}, containing
the non-prefix portage tree (easier to implement), so that
cross-emerge built against gx86 even on prefix-portage hosts.

One final thing I think we should really be looking at.  Recent gcc
toolchains have increasingly robust support for a "sysroot" concept
that achieves many of the same benefits we currently derive using
smoke-and-mirrors with ${ROOT} in cross-emerge.  Determining in which
versions various toolchain components start supporting sysroot
correctly, and using the feature (perhaps optionally) in those cases
(in lieu of the smoke-and-mirrors approach) would have the added
benefit of creating "relocatable" toolchains, which is to say,
rpath-free -- you can just stick 'em in a tarball and plop them down
wherever you like, and by-and-large, the right thing will actually
happen.  There is even early support for the same concept in recent
libtools, so that, hypothetically, you can build stuff within or
without your "SUBROOT" and everything will "just work" (although
package-level compatibility with the libtool feature is quite spotty,
I suspect).

I'd be curious to hear more about what your GSOC objectives are,
especially whether it's the "easy" or the "hard" fix, above, you are
shooting for.

-gmt