[gentoo-dev] [PMS] [PATCH v2] Correct the definition of ESYSROOT as EPREFIX isn't always applicable

[Ulrich Mueller]

Coming back to this old thread (from July 2019). After some discussion
in #gentoo-pms today, here is an updated version of the patch.

Cross-posting to gentoo-dev and CCing prefix again, to make sure that
everyone is on the same page.

Ulrich


From 712772b7ef5543693147d8f96c03189e810a6ee8 Mon Sep 17 00:00:00 2001
From: James Le Cuirot <chewi@g.o>
Date: Sun, 28 Jul 2019 22:37:35 +0100
Subject: [PATCH] Correct the definition of ESYSROOT as EPREFIX isn't always
 applicable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It was originally envisaged (but not stated in PMS) that SYSROOT would
only ever need to equal / or ROOT as a distinct SYSROOT would have no
benefit. A check was added to Portage to ensure this held. Myself, the
ChromiumOS team, and others have since been caught out by this check
when trying to bootstrap brand new systems from scratch. You cannot
bootstrap with no headers at all! The check will therefore be adjusted
to merely ensure that SYSROOT is / when ROOT is /.

There were differing assumptions about how prefixes applied to the
above. EPREFIX is traditionally something the user sets so some
thought that it would be applied to SYSROOT, regardless of the
latter's value. In order to honor the rule about there being no
distinct SYSROOT, this would mean that if SYSROOT is / then EPREFIX
would have to match BROOT. Despite that limitation, ESYSROOT was
written into PMS with a fixed value of ${SYSROOT}${EPREFIX}. Being
somewhat unfamiliar with prefix at the time, I didn't realise that
this view didn't align with what I'd had in mind and it was only when
I came to need a distinct SYSROOT that I realised there was a problem.

crossdev toolchains are installed to ${EPREFIX}/usr/${CHOST} but have
no further prefix appended and packages subsequently installed with
cross-emerge are placed in this location by setting ROOT. Bug #642604
recently revealed that the build system's prefix was being erroneously
duplicated on the end but I have now fixed this.

What if we want to bootstrap a brand new prefixed system using the
crossdev system as SYSROOT? This is the distinct SYSROOT case. The
problem is that there is no distinct variable for SYSROOT's prefix
and, as already stated, ESYSROOT is always ${SYSROOT}${EPREFIX}. We
therefore cannot do it! If the crossdev prefix is blank then ROOT's
must be blank too.

I also never intended to have the aforementioned limitation where
EPREFIX must match BROOT when SYSROOT is /. These are both entirely
artificial restrictions.

So how should it work instead? We originally intended for SYSROOT to
equal either / or ROOT so I imagined the prefix would automatically be
adjusted to match the prefix applicable at the matching location,
namely BROOT or EPREFIX. This is obviously more flexible than forcing
it to match EPREFIX.

What about the distinct SYSROOT case? With no distinct variable, we
have no way to explicitly set a prefix but this is likely only needed
when bootstrapping against crossdev systems, which are unprefixed by
nature. We therefore simply assume that the prefix is blank in this
case.

What about the cross-prefix case? Here, SYSROOT matches both / and
ROOT so which prefix do we choose? The bootstrap-prefix.sh script sets
flags to build against the target prefix so EPREFIX is used in this
case. This happens to fit the current definition of ESYSROOT anyway.

Legitimate concerns have been raised about building for a system with
a different prefix to the one you're building against. The only
binaries that leak from SYSROOT to ROOT are static libraries. Headers
from SYSROOT will obviously also influence how ROOT's binaries are
built. It is entirely possible that SYSROOT's prefix may leak through
a header but grepping /usr/include on my own main system reveals only
a few paths from a small handful of packages. pkg-config files
invariably include paths but these are almost always used at build
time, not runtime. A differing prefix would likely only occur in cases
involving core packages like the libc and kernel headers anyway. Also
consider that we have never prevented this from happening in the
past. It has always been possible to do "EPREFIX=/foo emerge bar" from
some system with a different prefix or no prefix at all. All we're
doing here is including the prefix (if any) in the ESYSROOT variable.

Should this warrant a new EAPI? I don't think so. All existing usage
of ESYSROOT that I have seen still fits with this new definition and
most of that usage has come from me. We're not even changing what the
variable is used for, just loosening the constraints around what it
can be set to.

If you have doubts about whether this makes sense or actually works in
practise, I have experimented with a prefixed system using all the
different combinations I could think of, including cross-compiling,
and it all worked as expected. Keep in mind that ESYSROOT is not magic
and currently isn't used very much. As such, neither the toolchain nor
pkg-config will use this sysroot if you don't explicitly tell them
to. For the former, I find CC="${CHOST}-gcc --sysroot=${ESYSROOT}"
works well. For the latter, crossdev installs a cross-pkg-config
wrapper but it is completely lacking prefix support at the moment. I
have fixes waiting on this change.

Signed-off-by: James Le Cuirot <chewi@g.o>
Signed-off-by: Ulrich Müller <ulm@g.o>
---
 dependencies.tex    | 16 +++++++++++++++-
 ebuild-env-vars.tex |  6 +++---
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/dependencies.tex b/dependencies.tex
index 3305b86..8bfe6f3 100644
--- a/dependencies.tex
+++ b/dependencies.tex
@@ -36,13 +36,27 @@
       \midrule
       Binary compatible with    & \t{CBUILD}      & \t{CHOST}          & \t{CHOST}         \\
       Base unprefixed path      & \t{/}           & \t{\$\{SYSROOT\}}  & \t{\$\{ROOT\}}    \\
-      Relevant offset-prefix    & \t{\$\{BROOT\}} & \t{\$\{EPREFIX\}}  & \t{\$\{EPREFIX\}} \\
+      Relevant offset-prefix    & \t{\$\{BROOT\}} & See table~\ref{tab:depend-prefix}
+                                                                       & \t{\$\{EPREFIX\}} \\
       Path combined with prefix & \t{\$\{BROOT\}} & \t{\$\{ESYSROOT\}} & \t{\$\{EROOT\}}   \\
       PM query command option   & \t{-b}          & \t{-d}             & \t{-r}            \\
       \bottomrule
     \end{tabular}
 \end{centertable}
 
+\begin{centertable}{Prefix values for \t{DEPEND}}
+    \label{tab:depend-prefix}
+    \begin{tabular}{llll}
+      \toprule
+      If \t{SYSROOT} is:     & \t{\$\{ROOT\}}    & Empty but not equal to \t{\$\{ROOT\}}
+                                                                   & Other             \\
+      \midrule
+      Then offset-prefix is: & \t{\$\{EPREFIX\}} & \t{\$\{BROOT\}} & Blank             \\
+      And \t{ESYSROOT} is:   & \t{\$\{EROOT\}}   & \t{\$\{BROOT\}} & \t{\$\{SYSROOT\}} \\
+      \bottomrule
+    \end{tabular}
+\end{centertable}
+
 There are three classes of dependencies supported by ebuilds:
 
 \begin{compactitem}
diff --git a/ebuild-env-vars.tex b/ebuild-env-vars.tex
index ae8bd7d..117a19d 100644
--- a/ebuild-env-vars.tex
+++ b/ebuild-env-vars.tex
@@ -141,9 +141,9 @@ variable.
 \t{ESYSROOT} &
     Ditto &
     No &
-    Contains the concatenation of the paths in the \t{SYSROOT} and \t{EPREFIX} variables,
-    for convenience. See also the \t{EPREFIX} variable. Only for EAPIs listed
-    in table~\ref{tab:offset-env-vars-table} as supporting \t{ESYSROOT}. \\
+    Contains the concatenation of the \t{SYSROOT} path and applicable prefix value, as determined
+    by table~\ref{tab:depend-prefix}. Only for EAPIs listed in table~\ref{tab:offset-env-vars-table}
+    as supporting \t{ESYSROOT}. \\
 \t{BROOT} &
     Ditto &
     No &
-- 
2.27.0