[gentoo-catalyst] [PATCH 01/11] targets: Remove unused 'enter' command

[gentoo-catalyst] [PATCH 01/11] targets: Remove unused 'enter' command

[Matt Turner]

Doesn't appear to have ever been wired up. I'm going to add something
like what I think this was, but without the duplication.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 targets/embedded/controller.sh | 3 ---
 targets/stage1/controller.sh   | 3 ---
 targets/stage2/controller.sh   | 4 ----
 targets/stage3/controller.sh   | 4 ----
 targets/stage4/controller.sh   | 4 ----
 5 files changed, 18 deletions(-)

diff --git a/targets/embedded/controller.sh b/targets/embedded/controller.sh
index 48867897..c2e5994c 100755
--- a/targets/embedded/controller.sh
+++ b/targets/embedded/controller.sh
@@ -3,9 +3,6 @@
 source ${clst_shdir}/support/functions.sh
 
 case ${1} in
-	enter)
-	;;
-
 	build_packages)
 		shift
 		export clst_packages="$*"
diff --git a/targets/stage1/controller.sh b/targets/stage1/controller.sh
index ab127114..0db1614d 100755
--- a/targets/stage1/controller.sh
+++ b/targets/stage1/controller.sh
@@ -3,9 +3,6 @@
 source "${clst_shdir}/support/functions.sh"
 
 case "$1" in
-	enter)
-	;;
-
 	run)
 		cp "${clst_shdir}/stage1/build.py" "${clst_chroot_path}/tmp"
 
diff --git a/targets/stage2/controller.sh b/targets/stage2/controller.sh
index efa57648..fa5592e1 100755
--- a/targets/stage2/controller.sh
+++ b/targets/stage2/controller.sh
@@ -5,10 +5,6 @@ source ${clst_shdir}/support/functions.sh
 # Only put commands in this section that you want every target to execute.
 # This is a global default file and will affect every target
 case $1 in
-	enter)
-		${clst_CHROOT} ${clst_chroot_path}
-	;;
-
 	run)
 		shift
 		export clst_packages="$*"
diff --git a/targets/stage3/controller.sh b/targets/stage3/controller.sh
index f1ca6883..f4a3c7a1 100755
--- a/targets/stage3/controller.sh
+++ b/targets/stage3/controller.sh
@@ -5,10 +5,6 @@ source ${clst_shdir}/support/functions.sh
 # Only put commands in this section that you want every target to execute.
 # This is a global default file and will affect every target
 case $1 in
-	enter)
-		${clst_CHROOT} ${clst_chroot_path}
-	;;
-
 	run)
 		shift
 		export clst_packages="$*"
diff --git a/targets/stage4/controller.sh b/targets/stage4/controller.sh
index ba0774d1..ee078f60 100755
--- a/targets/stage4/controller.sh
+++ b/targets/stage4/controller.sh
@@ -5,10 +5,6 @@ source ${clst_shdir}/support/functions.sh
 # Only put commands in this section that you want every target to execute.
 # This is a global default file and will affect every target
 case $1 in
-	enter)
-		${clst_CHROOT} ${clst_chroot_path}
-	;;
-
 	pre-kmerge)
 		# Sets up the build environment before any kernels are compiled
 		exec_in_chroot ${clst_shdir}/support/pre-kmerge.sh
-- 
2.26.2

[gentoo-catalyst] [PATCH 02/11] targets: Remove some obvious comments

[Matt Turner]

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 targets/stage2/controller.sh | 2 --
 targets/stage3/controller.sh | 2 --
 targets/stage4/controller.sh | 2 --
 3 files changed, 6 deletions(-)

diff --git a/targets/stage2/controller.sh b/targets/stage2/controller.sh
index fa5592e1..8ee51a5b 100755
--- a/targets/stage2/controller.sh
+++ b/targets/stage2/controller.sh
@@ -2,8 +2,6 @@
 
 source ${clst_shdir}/support/functions.sh
 
-# Only put commands in this section that you want every target to execute.
-# This is a global default file and will affect every target
 case $1 in
 	run)
 		shift
diff --git a/targets/stage3/controller.sh b/targets/stage3/controller.sh
index f4a3c7a1..63f245a8 100755
--- a/targets/stage3/controller.sh
+++ b/targets/stage3/controller.sh
@@ -2,8 +2,6 @@
 
 source ${clst_shdir}/support/functions.sh
 
-# Only put commands in this section that you want every target to execute.
-# This is a global default file and will affect every target
 case $1 in
 	run)
 		shift
diff --git a/targets/stage4/controller.sh b/targets/stage4/controller.sh
index ee078f60..0c40145e 100755
--- a/targets/stage4/controller.sh
+++ b/targets/stage4/controller.sh
@@ -2,8 +2,6 @@
 
 source ${clst_shdir}/support/functions.sh
 
-# Only put commands in this section that you want every target to execute.
-# This is a global default file and will affect every target
 case $1 in
 	pre-kmerge)
 		# Sets up the build environment before any kernels are compiled
-- 
2.26.2

[gentoo-catalyst] [PATCH 03/11] targets: Update seed stage's sys-apps/portage

[Matt Turner]

We'll want to use some new features that have been added specifically
for catalyst. It's probably a good idea to use the latest stable portage
anyway.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 targets/stage1/chroot.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/targets/stage1/chroot.sh b/targets/stage1/chroot.sh
index 50afef4a..1085641e 100755
--- a/targets/stage1/chroot.sh
+++ b/targets/stage1/chroot.sh
@@ -32,6 +32,9 @@ FEATURES="${FEATURES} nodoc noman noinfo"
 if [ -n "${clst_update_seed}" ]; then
 	if [ "${clst_update_seed}" == "yes" ]; then
 		echo "Updating seed stage..."
+
+		ROOT=/ USE=-rsync-verify run_merge --oneshot --update sys-apps/portage
+
 		if [ -n "${clst_update_seed_command}" ]; then
 			ROOT=/ run_merge --buildpkg=n "${clst_update_seed_command}"
 		else
-- 
2.26.2

[gentoo-catalyst] [PATCH 04/11] targets: Update BINPKG_COMPRESS to new zstd default

[Matt Turner]

In portage commit 230595cf600c (Default BINPKG_COMPRESSION to zstd (bug
715108)) the default BINPKG_COMPRESSION setting was changed to zstd. But
to retain compatibility with binpkg consumers, the default is unchanged
for existing installations, so we need to update our BINPKG_COMPRESSION.

Bug: https://bugs.gentoo.org/715108
Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 targets/stage1/chroot.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/targets/stage1/chroot.sh b/targets/stage1/chroot.sh
index 1085641e..e5a3e0b0 100755
--- a/targets/stage1/chroot.sh
+++ b/targets/stage1/chroot.sh
@@ -24,6 +24,9 @@ BOOTSTRAP_USE="$(portageq envvar BOOTSTRAP_USE)"
 
 FEATURES="${FEATURES} nodoc noman noinfo"
 
+sed -i -e 's:BINPKG_COMPRESS="bzip2":BINPKG_COMPRESS="zstd":' \
+	/usr/share/portage/config/make.globals
+
 # We need to ensure the base stage3 has USE="bindist"
 # if BINDIST is set to avoid issues with openssl / openssh
 [ -e ${clst_make_conf} ] && echo "USE=\"${BINDIST} ${USE}\"" >> ${clst_make_conf}
-- 
2.26.2

[gentoo-catalyst] [PATCH 05/11] targets: Update the @changed-subslot set by default

[Matt Turner]

In portage commit 1789fdf2ee81 (Add @changed-subslot package set) I
added this: the set of upgradable packages for which the highest visible
version has a different subslot than the currently installed version.

Updating the entire stage is expensive and unnecessary (since we're
going to build the latest packages in stage1 and then rebuild everything
in stage3).

What we definitely do need to update in the original stage3 however, is
any package that would trigger a subslot rebuild.

For example: gcc links with libmpfr.so from dev-libs/mpfr. mpfr's SONAME
changes from libmpfr.so.4 (SLOT="0/4") to libmpfr.so.6 (SLOT="0/6"). If
the seed stage's dev-libs/mpfr is not updated before emerging gcc, gcc
will link with libmpfr.so.4, but the latest version of dev-libs/mpfr
will be built and libmpfr.so.6 included into the stage1. Since the old
libmpfr.so.4 is not included in the stage1, gcc will not work, breaking
subsequent stage builds.

Our current options to update the seed are too large a hammer (e.g.,
"--update --deep --newuse @world" or "--update --deep --newuse
--complete-graph --rebuild-if-new-ver gcc") and spend too much time
updating seed stages for no gain beyond updating only packages for whom
the subslot has changed.

Bug: https://bugs.gentoo.org/739004
Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 targets/stage1/chroot.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/targets/stage1/chroot.sh b/targets/stage1/chroot.sh
index e5a3e0b0..a1818425 100755
--- a/targets/stage1/chroot.sh
+++ b/targets/stage1/chroot.sh
@@ -40,6 +40,8 @@ if [ -n "${clst_update_seed}" ]; then
 
 		if [ -n "${clst_update_seed_command}" ]; then
 			ROOT=/ run_merge --buildpkg=n "${clst_update_seed_command}"
+		elif grep -q '^\[changed-subslot\]' /usr/share/portage/config/sets/portage.conf; then
+			ROOT=/ run_merge --ignore-built-slot-operator-deps y @changed-subslot
 		else
 			ROOT=/ run_merge --update --deep --newuse --complete-graph --rebuild-if-new-ver gcc
 		fi
-- 
2.26.2

[gentoo-catalyst] [PATCH 06/11] catalyst: Store references to functions

[Matt Turner]

... rather than their names. This makes it possible for tooling to
understand the code structure better.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/base/stagebase.py        | 69 ++++++++++++++++---------------
 catalyst/targets/embedded.py      | 34 +++++++--------
 catalyst/targets/livecd_stage1.py | 18 ++++----
 catalyst/targets/livecd_stage2.py | 46 ++++++++++-----------
 catalyst/targets/netboot.py       | 34 +++++++--------
 catalyst/targets/stage1.py        | 12 +++---
 catalyst/targets/stage4.py        | 36 ++++++++--------
 7 files changed, 125 insertions(+), 124 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index ed4d1227..447e073d 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -30,6 +30,23 @@ from catalyst.fileops import ensure_dirs, clear_dir, clear_path
 from catalyst.base.resume import AutoResume
 
 
+def run_sequence(sequence):
+    for func in sequence:
+        log.notice('--- Running action sequence: %s', func.__name__)
+        sys.stdout.flush()
+        try:
+            func()
+        except LockInUse:
+            log.error('Unable to aquire the lock...')
+            return False
+        except Exception:
+            log.error('Exception running action sequence %s',
+                      func.__name__, exc_info=True)
+            return False
+
+    return True
+
+
 class StageBase(TargetBase, ClearBase, GenBase):
     """
     This class does all of the chroot setup, copying of files, etc. It is
@@ -475,39 +492,39 @@ class StageBase(TargetBase, ClearBase, GenBase):
         Or it calls the normal set_action_sequence() for the target stage.
         """
         if "purgeonly" in self.settings["options"]:
-            self.build_sequence.append("remove_chroot")
+            self.build_sequence.append(self.remove_chroot)
             return
         self.set_action_sequence()
 
     def set_action_sequence(self):
         """Set basic stage1, 2, 3 action sequences"""
         self.prepare_sequence.extend([
-            "unpack",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "run_local",
-            "preclean",
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.run_local,
+            self.preclean,
         ])
         self.finish_sequence.extend([
-            "clean",
+            self.clean,
         ])
         self.set_completion_action_sequences()
 
     def set_completion_action_sequences(self):
         if "fetch" not in self.settings["options"]:
-            self.finish_sequence.append("capture")
+            self.finish_sequence.append(self.capture)
         if "keepwork" in self.settings["options"]:
-            self.finish_sequence.append("clear_autoresume")
+            self.finish_sequence.append(self.clear_autoresume)
         elif "seedcache" in self.settings["options"]:
-            self.finish_sequence.append("remove_autoresume")
+            self.finish_sequence.append(self.remove_autoresume)
         else:
-            self.finish_sequence.append("remove_autoresume")
-            self.finish_sequence.append("remove_chroot")
+            self.finish_sequence.append(self.remove_autoresume)
+            self.finish_sequence.append(self.remove_chroot)
 
     def set_use(self):
         use = self.settings["spec_prefix"] + "/use"
@@ -1308,22 +1325,6 @@ class StageBase(TargetBase, ClearBase, GenBase):
 
         log.debug('setup_environment(); env = %r', self.env)
 
-    def run_sequence(self, sequence):
-        for func in sequence:
-            log.notice('--- Running action sequence: %s', func)
-            sys.stdout.flush()
-            try:
-                getattr(self, func)()
-            except LockInUse:
-                log.error('Unable to aquire the lock...')
-                return False
-            except Exception:
-                log.error('Exception running action sequence %s',
-                          func, exc_info=True)
-                return False
-
-        return True
-
     def run(self):
         self.chroot_lock.write_lock()
 
@@ -1342,14 +1343,14 @@ class StageBase(TargetBase, ClearBase, GenBase):
             log.info('StageBase: run() purge')
             self.purge()
 
-        if not self.run_sequence(self.prepare_sequence):
+        if not run_sequence(self.prepare_sequence):
             return False
 
         with namespace(mount=True):
-            if not self.run_sequence(self.build_sequence):
+            if not run_sequence(self.build_sequence):
                 return False
 
-        if not self.run_sequence(self.finish_sequence):
+        if not run_sequence(self.finish_sequence):
             return False
 
         return True
diff --git a/catalyst/targets/embedded.py b/catalyst/targets/embedded.py
index 918b36bf..7d328808 100644
--- a/catalyst/targets/embedded.py
+++ b/catalyst/targets/embedded.py
@@ -42,27 +42,27 @@ class embedded(StageBase):
 
     def set_action_sequence(self):
         self.prepare_sequence.extend([
-            "unpack",
-            "config_profile_link",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "build_kernel",
-            "build_packages",
-            "root_overlay",
-            "fsscript",
-            "unmerge",
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.build_kernel,
+            self.build_packages,
+            self.root_overlay,
+            self.fsscript,
+            self.unmerge,
         ])
         self.finish_sequence.extend([
-            "remove",
-            "empty",
-            "clean",
-            "capture",
-            "clear_autoresume",
+            self.remove,
+            self.empty,
+            self.clean,
+            self.capture,
+            self.clear_autoresume,
         ])
 
     def set_root_path(self):
diff --git a/catalyst/targets/livecd_stage1.py b/catalyst/targets/livecd_stage1.py
index 5c5e9f58..eb6bb0f0 100644
--- a/catalyst/targets/livecd_stage1.py
+++ b/catalyst/targets/livecd_stage1.py
@@ -24,19 +24,19 @@ class livecd_stage1(StageBase):
 
     def set_action_sequence(self):
         self.prepare_sequence.extend([
-            "unpack",
-            "config_profile_link",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "build_packages",
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.build_packages,
         ])
         self.finish_sequence.extend([
-            "clean",
+            self.clean,
         ])
         self.set_completion_action_sequences()
 
diff --git a/catalyst/targets/livecd_stage2.py b/catalyst/targets/livecd_stage2.py
index 3606047f..5fecff85 100644
--- a/catalyst/targets/livecd_stage2.py
+++ b/catalyst/targets/livecd_stage2.py
@@ -87,34 +87,34 @@ class livecd_stage2(StageBase):
 
     def set_action_sequence(self):
         self.prepare_sequence.extend([
-            "unpack",
-            "config_profile_link",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "run_local",
-            "build_kernel"
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.run_local,
+            self.build_kernel
         ])
         if "fetch" not in self.settings["options"]:
             self.build_sequence.extend([
-                "bootloader",
-                "preclean",
-                "livecd_update",
-                "root_overlay",
-                "fsscript",
-                "rcupdate",
-                "unmerge",
+                self.bootloader,
+                self.preclean,
+                self.livecd_update,
+                self.root_overlay,
+                self.fsscript,
+                self.rcupdate,
+                self.unmerge,
             ])
             self.finish_sequence.extend([
-                "remove",
-                "empty",
-                "clean",
-                "target_setup",
-                "setup_overlay",
-                "create_iso",
+                self.remove,
+                self.empty,
+                self.clean,
+                self.target_setup,
+                self.setup_overlay,
+                self.create_iso,
             ])
-        self.finish_sequence.append("clear_autoresume")
+        self.finish_sequence.append(self.clear_autoresume)
diff --git a/catalyst/targets/netboot.py b/catalyst/targets/netboot.py
index 55f4dff1..9a7e59e5 100644
--- a/catalyst/targets/netboot.py
+++ b/catalyst/targets/netboot.py
@@ -159,25 +159,25 @@ class netboot(StageBase):
 
     def set_action_sequence(self):
         self.prepare_sequence.extend([
-            "unpack",
-            "config_profile_link",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "build_packages",
-            "root_overlay",
-            "copy_files_to_image",
-            "setup_overlay",
-            "build_kernel",
-            "move_kernels",
-            "remove",
-            "empty",
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.build_packages,
+            self.root_overlay,
+            self.copy_files_to_image,
+            self.setup_overlay,
+            self.build_kernel,
+            self.move_kernels,
+            self.remove,
+            self.empty,
         ])
         self.finish_sequence.extend([
-            "clean",
-            "clear_autoresume",
+            self.clean,
+            self.clear_autoresume,
         ])
diff --git a/catalyst/targets/stage1.py b/catalyst/targets/stage1.py
index 5aa27bb9..5a154e76 100644
--- a/catalyst/targets/stage1.py
+++ b/catalyst/targets/stage1.py
@@ -82,15 +82,15 @@ class stage1(StageBase):
         chroot for re-use in stage2 without the need to unpack it.
         '''
         if "fetch" not in self.settings["options"]:
-            self.finish_sequence.append("capture")
+            self.finish_sequence.append(self.capture)
         if "keepwork" in self.settings["options"]:
-            self.finish_sequence.append("clear_autoresume")
+            self.finish_sequence.append(self.clear_autoresume)
         elif "seedcache" in self.settings["options"]:
-            self.finish_sequence.append("remove_autoresume")
-            self.finish_sequence.append("clean_stage1")
+            self.finish_sequence.append(self.remove_autoresume)
+            self.finish_sequence.append(self.clean_stage1)
         else:
-            self.finish_sequence.append("remove_autoresume")
-            self.finish_sequence.append("remove_chroot")
+            self.finish_sequence.append(self.remove_autoresume)
+            self.finish_sequence.append(self.remove_chroot)
 
     def clean_stage1(self):
         '''seedcache is enabled, so salvage the /tmp/stage1root,
diff --git a/catalyst/targets/stage4.py b/catalyst/targets/stage4.py
index b7f74b01..997139a5 100644
--- a/catalyst/targets/stage4.py
+++ b/catalyst/targets/stage4.py
@@ -40,27 +40,27 @@ class stage4(StageBase):
 
     def set_action_sequence(self):
         self.prepare_sequence.extend([
-            "unpack",
-            "config_profile_link",
-            "setup_confdir",
-            "portage_overlay",
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
         ])
         self.build_sequence.extend([
-            "bind",
-            "chroot_setup",
-            "setup_environment",
-            "build_packages",
-            "build_kernel",
-            "bootloader",
-            "root_overlay",
-            "fsscript",
-            "preclean",
-            "rcupdate",
-            "unmerge",
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+            self.build_packages,
+            self.build_kernel,
+            self.bootloader,
+            self.root_overlay,
+            self.fsscript,
+            self.preclean,
+            self.rcupdate,
+            self.unmerge,
         ])
         self.finish_sequence.extend([
-            "remove",
-            "empty",
-            "clean",
+            self.remove,
+            self.empty,
+            self.clean,
         ])
         self.set_completion_action_sequences()
-- 
2.26.2

[gentoo-catalyst] [PATCH 07/11] catalyst: Call config_profile_link for all targets

[Matt Turner]

This reverts the following two commits, which have no documented
rationale.

Revert "Set the profile by calling eselect."

This reverts commit 90c03f9dc255ba89849e46490f9ead7ab3921950.

Revert "Drop config_profile_link from the action_sequence for the generic stage."

This reverts commit 3bd10159bf7cfe14b6d8a8218b94eca73be4c997.

Doing so improves the code in two ways:

   1) it makes prepare_sequence identical across all targets, which will
      allow deduplicating some code
   2) it no longer calls eselect profile each time chroot-functions.sh
      is sourced (even for those targets that were still calling
      config_profile_link)

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/base/stagebase.py          | 3 ++-
 targets/support/chroot-functions.sh | 3 ---
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 447e073d..53b0a224 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -500,6 +500,7 @@ class StageBase(TargetBase, ClearBase, GenBase):
         """Set basic stage1, 2, 3 action sequences"""
         self.prepare_sequence.extend([
             self.unpack,
+            self.config_profile_link,
             self.setup_confdir,
             self.portage_overlay,
         ])
@@ -771,7 +772,7 @@ class StageBase(TargetBase, ClearBase, GenBase):
         log.info('Configuring profile link...')
         make_profile = Path(self.settings['chroot_path'] + self.settings['port_conf'],
                             'make.profile')
-        make_profile.unlink()
+        make_profile.unlink(missing_ok=True)
         make_profile.symlink_to(Path('../..' + self.settings['repo_basedir'],
                                      self.settings['repo_name'],
                                      'profiles',
diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
index 2aec018e..88465c31 100755
--- a/targets/support/chroot-functions.sh
+++ b/targets/support/chroot-functions.sh
@@ -1,8 +1,5 @@
 #!/bin/bash
 
-# Set the profile
-eselect profile set ${clst_target_profile}
-
 # Trap these signals and kill ourselves if received
 # Force ourselves to die if any of these signals are received
 # most likely our controlling terminal is gone
-- 
2.26.2

[gentoo-catalyst] [PATCH 08/11] catalyst: Deduplicate prepare_sequence assignments

[Matt Turner]

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/base/stagebase.py        | 13 ++++++-------
 catalyst/targets/embedded.py      |  6 ------
 catalyst/targets/livecd_stage1.py |  6 ------
 catalyst/targets/livecd_stage2.py |  6 ------
 catalyst/targets/netboot.py       |  6 ------
 catalyst/targets/stage4.py        |  6 ------
 6 files changed, 6 insertions(+), 37 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 53b0a224..3b8d3a6e 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -84,7 +84,12 @@ class StageBase(TargetBase, ClearBase, GenBase):
             "portage_overlay",
             "portage_prefix",
         ])
-        self.prepare_sequence = []
+        self.prepare_sequence = [
+            self.unpack,
+            self.config_profile_link,
+            self.setup_confdir,
+            self.portage_overlay,
+        ]
         self.build_sequence = []
         self.finish_sequence = []
 
@@ -498,12 +503,6 @@ class StageBase(TargetBase, ClearBase, GenBase):
 
     def set_action_sequence(self):
         """Set basic stage1, 2, 3 action sequences"""
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
diff --git a/catalyst/targets/embedded.py b/catalyst/targets/embedded.py
index 7d328808..83b17ad2 100644
--- a/catalyst/targets/embedded.py
+++ b/catalyst/targets/embedded.py
@@ -41,12 +41,6 @@ class embedded(StageBase):
         StageBase.__init__(self, spec, addlargs)
 
     def set_action_sequence(self):
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
diff --git a/catalyst/targets/livecd_stage1.py b/catalyst/targets/livecd_stage1.py
index eb6bb0f0..9fe95a00 100644
--- a/catalyst/targets/livecd_stage1.py
+++ b/catalyst/targets/livecd_stage1.py
@@ -23,12 +23,6 @@ class livecd_stage1(StageBase):
         StageBase.__init__(self, spec, addlargs)
 
     def set_action_sequence(self):
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
diff --git a/catalyst/targets/livecd_stage2.py b/catalyst/targets/livecd_stage2.py
index 5fecff85..ab8f4a0e 100644
--- a/catalyst/targets/livecd_stage2.py
+++ b/catalyst/targets/livecd_stage2.py
@@ -86,12 +86,6 @@ class livecd_stage2(StageBase):
                                     print_traceback=True)
 
     def set_action_sequence(self):
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
diff --git a/catalyst/targets/netboot.py b/catalyst/targets/netboot.py
index 9a7e59e5..cb5e7ae4 100644
--- a/catalyst/targets/netboot.py
+++ b/catalyst/targets/netboot.py
@@ -158,12 +158,6 @@ class netboot(StageBase):
         self.resume.enable("empty")
 
     def set_action_sequence(self):
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
diff --git a/catalyst/targets/stage4.py b/catalyst/targets/stage4.py
index 997139a5..9e53f070 100644
--- a/catalyst/targets/stage4.py
+++ b/catalyst/targets/stage4.py
@@ -39,12 +39,6 @@ class stage4(StageBase):
         self.settings["cleanables"].remove('/etc/resolv.conf')
 
     def set_action_sequence(self):
-        self.prepare_sequence.extend([
-            self.unpack,
-            self.config_profile_link,
-            self.setup_confdir,
-            self.portage_overlay,
-        ])
         self.build_sequence.extend([
             self.bind,
             self.chroot_setup,
-- 
2.26.2

[gentoo-catalyst] [PATCH 09/11] catalyst: Deduplicate the common build_sequence steps

[Matt Turner]

This also provides a single location to add a function to all targets'
build_sequence.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/base/stagebase.py        | 11 ++++++-----
 catalyst/targets/embedded.py      |  3 ---
 catalyst/targets/livecd_stage1.py |  3 ---
 catalyst/targets/livecd_stage2.py |  3 ---
 catalyst/targets/netboot.py       |  3 ---
 catalyst/targets/stage4.py        |  3 ---
 6 files changed, 6 insertions(+), 20 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 3b8d3a6e..40b60af3 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -90,7 +90,11 @@ class StageBase(TargetBase, ClearBase, GenBase):
             self.setup_confdir,
             self.portage_overlay,
         ]
-        self.build_sequence = []
+        self.build_sequence = [
+            self.bind,
+            self.chroot_setup,
+            self.setup_environment,
+        ]
         self.finish_sequence = []
 
         self.set_valid_build_kernel_vars(addlargs)
@@ -497,16 +501,13 @@ class StageBase(TargetBase, ClearBase, GenBase):
         Or it calls the normal set_action_sequence() for the target stage.
         """
         if "purgeonly" in self.settings["options"]:
-            self.build_sequence.append(self.remove_chroot)
+            self.build_sequence = [self.remove_chroot]
             return
         self.set_action_sequence()
 
     def set_action_sequence(self):
         """Set basic stage1, 2, 3 action sequences"""
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.run_local,
             self.preclean,
         ])
diff --git a/catalyst/targets/embedded.py b/catalyst/targets/embedded.py
index 83b17ad2..b677b226 100644
--- a/catalyst/targets/embedded.py
+++ b/catalyst/targets/embedded.py
@@ -42,9 +42,6 @@ class embedded(StageBase):
 
     def set_action_sequence(self):
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.build_kernel,
             self.build_packages,
             self.root_overlay,
diff --git a/catalyst/targets/livecd_stage1.py b/catalyst/targets/livecd_stage1.py
index 9fe95a00..dbfa54ed 100644
--- a/catalyst/targets/livecd_stage1.py
+++ b/catalyst/targets/livecd_stage1.py
@@ -24,9 +24,6 @@ class livecd_stage1(StageBase):
 
     def set_action_sequence(self):
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.build_packages,
         ])
         self.finish_sequence.extend([
diff --git a/catalyst/targets/livecd_stage2.py b/catalyst/targets/livecd_stage2.py
index ab8f4a0e..e90e9f53 100644
--- a/catalyst/targets/livecd_stage2.py
+++ b/catalyst/targets/livecd_stage2.py
@@ -87,9 +87,6 @@ class livecd_stage2(StageBase):
 
     def set_action_sequence(self):
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.run_local,
             self.build_kernel
         ])
diff --git a/catalyst/targets/netboot.py b/catalyst/targets/netboot.py
index cb5e7ae4..a2a9fcb3 100644
--- a/catalyst/targets/netboot.py
+++ b/catalyst/targets/netboot.py
@@ -159,9 +159,6 @@ class netboot(StageBase):
 
     def set_action_sequence(self):
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.build_packages,
             self.root_overlay,
             self.copy_files_to_image,
diff --git a/catalyst/targets/stage4.py b/catalyst/targets/stage4.py
index 9e53f070..f8383f75 100644
--- a/catalyst/targets/stage4.py
+++ b/catalyst/targets/stage4.py
@@ -40,9 +40,6 @@ class stage4(StageBase):
 
     def set_action_sequence(self):
         self.build_sequence.extend([
-            self.bind,
-            self.chroot_setup,
-            self.setup_environment,
             self.build_packages,
             self.build_kernel,
             self.bootloader,
-- 
2.26.2

[gentoo-catalyst] [PATCH 10/11] catalyst: Add option to enter the chroot before building

[Matt Turner]

With --enter-chroot, after the mounts and environment are set up,
catalyst will drop you into a shell inside the chroot. Useful for
hacking or debugging.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/base/stagebase.py | 17 ++++++++++++++++-
 catalyst/main.py           |  4 ++++
 doc/catalyst.1.txt         |  3 +++
 3 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 40b60af3..02e202c1 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -20,7 +20,7 @@ from catalyst import log
 from catalyst.context import namespace
 from catalyst.defaults import (confdefaults, MOUNT_DEFAULTS, PORT_LOGDIR_CLEAN)
 from catalyst.support import (CatalystError, file_locate, normpath,
-                              cmd, read_makeconf, get_repo_name, ismount,
+                              cmd, command, read_makeconf, get_repo_name,
                               file_check, sanitize_name)
 from catalyst.base.targetbase import TargetBase
 from catalyst.base.clearbase import ClearBase
@@ -94,6 +94,7 @@ class StageBase(TargetBase, ClearBase, GenBase):
             self.bind,
             self.chroot_setup,
             self.setup_environment,
+            self.enter_chroot,
         ]
         self.finish_sequence = []
 
@@ -1326,6 +1327,20 @@ class StageBase(TargetBase, ClearBase, GenBase):
 
         log.debug('setup_environment(); env = %r', self.env)
 
+    def enter_chroot(self):
+        if 'enter-chroot' not in self.settings['options']:
+            return
+
+        chroot = command('chroot')
+        bash = command('bash')
+
+        log.notice("Entering chroot")
+        try:
+            cmd([chroot, self.settings['chroot_path'], bash, '-l'],
+                env=self.env)
+        except CatalystError:
+            pass
+
     def run(self):
         self.chroot_lock.write_lock()
 
diff --git a/catalyst/main.py b/catalyst/main.py
index 48daf004..b0d9015f 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -120,6 +120,8 @@ def get_parser():
     parser.add_argument('-V', '--version',
                         action='version', version=get_version(),
                         help='display version information')
+    parser.add_argument('--enter-chroot', default=False, action='store_true',
+                        help='Enter chroot before starting the build')
 
     group = parser.add_argument_group('Program output options')
     group.add_argument('-d', '--debug',
@@ -293,6 +295,8 @@ def _main(parser, opts):
         options.append('purgetmponly')
     if opts.clear_autoresume:
         options.append('clear-autoresume')
+    if opts.enter_chroot:
+        options.append('enter-chroot')
 
     # Make sure we have some work before moving further.
     if not myspecfile and not mycmdline:
diff --git a/doc/catalyst.1.txt b/doc/catalyst.1.txt
index 90d5a24b..217fc86a 100644
--- a/doc/catalyst.1.txt
+++ b/doc/catalyst.1.txt
@@ -39,6 +39,9 @@ configuration file is installed at '/etc/catalyst/catalyst.conf'.
 *-d*::
 Enable debugging mode
 
+*--enter-chroot*::
+Enter the chroot before starting the build.
+
 *--fetchonly*::
 *-F*::
 This tells *catalyst* to only fetch distfiles for the given packages without
-- 
2.26.2

[gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Matt Turner]

There should be no need for custom updates, and if there are exceptional
circumstances the new --enter-chroot option can be used.

Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
 catalyst/targets/stage1.py          |  2 --
 doc/catalyst-spec.5.txt             | 10 -------
 targets/stage1/chroot.sh            | 43 +++++++++--------------------
 targets/support/chroot-functions.sh |  2 +-
 4 files changed, 14 insertions(+), 43 deletions(-)

diff --git a/catalyst/targets/stage1.py b/catalyst/targets/stage1.py
index 5a154e76..f7c723ee 100644
--- a/catalyst/targets/stage1.py
+++ b/catalyst/targets/stage1.py
@@ -16,8 +16,6 @@ class stage1(StageBase):
     required_values = frozenset()
     valid_values = required_values | frozenset([
         "chost",
-        "update_seed",
-        "update_seed_command",
     ])
 
     def __init__(self, spec, addlargs):
diff --git a/doc/catalyst-spec.5.txt b/doc/catalyst-spec.5.txt
index 4c1df857..47a62709 100644
--- a/doc/catalyst-spec.5.txt
+++ b/doc/catalyst-spec.5.txt
@@ -116,16 +116,6 @@ releases, we use a default README.txt, and this will be used on your
 CD if you do not provide one yourself.  We do not use this for the
 official releases.  This setting is supported by the livecd targets.
 
-*update_seed*::
-This is an optional setting supported by stage1 to tell catalyst if
-it should update the seed stage or not (valid values: `yes no`).
-
-*update_seed_command*::
-This is an optional command to pass to emerge for updating the seed
-stage (example: `--update dev-libs/mpfr dev-libs/mpc dev-libs/gmp`)
-If not specified, catalyst will update gcc deps.
-This setting requires enabling update_seed.
-
 Compilation
 ~~~~~~~~~~~
 
diff --git a/targets/stage1/chroot.sh b/targets/stage1/chroot.sh
index a1818425..76a30941 100755
--- a/targets/stage1/chroot.sh
+++ b/targets/stage1/chroot.sh
@@ -27,37 +27,20 @@ FEATURES="${FEATURES} nodoc noman noinfo"
 sed -i -e 's:BINPKG_COMPRESS="bzip2":BINPKG_COMPRESS="zstd":' \
 	/usr/share/portage/config/make.globals
 
-# We need to ensure the base stage3 has USE="bindist"
-# if BINDIST is set to avoid issues with openssl / openssh
-[ -e ${clst_make_conf} ] && echo "USE=\"${BINDIST} ${USE}\"" >> ${clst_make_conf}
-
-# Update stage3
-if [ -n "${clst_update_seed}" ]; then
-	if [ "${clst_update_seed}" == "yes" ]; then
-		echo "Updating seed stage..."
-
-		ROOT=/ USE=-rsync-verify run_merge --oneshot --update sys-apps/portage
-
-		if [ -n "${clst_update_seed_command}" ]; then
-			ROOT=/ run_merge --buildpkg=n "${clst_update_seed_command}"
-		elif grep -q '^\[changed-subslot\]' /usr/share/portage/config/sets/portage.conf; then
-			ROOT=/ run_merge --ignore-built-slot-operator-deps y @changed-subslot
-		else
-			ROOT=/ run_merge --update --deep --newuse --complete-graph --rebuild-if-new-ver gcc
-		fi
-	elif [ "${clst_update_seed}" != "no" ]; then
-		echo "Invalid setting for update_seed: ${clst_update_seed}"
-		exit 1
+# Update seed stage
+echo "Updating seed stage"
+(
+	# Don't build or use binpkgs when updating the seed stage
+	clst_PKGCACHE= setup_emerge_opts
+
+	export ROOT=/
+	USE=-rsync-verify run_merge --oneshot --update sys-apps/portage
+	if grep -q '^\[changed-subslot\]' /usr/share/portage/config/sets/portage.conf; then
+		run_merge --ignore-built-slot-operator-deps y @changed-subslot
+	else
+		run_merge --update --deep --newuse --complete-graph --rebuild-if-new-ver gcc
 	fi
-
-	# reset emerge options for the target
-	clst_update_seed=no setup_emerge_opts
-else
-	echo "Skipping seed stage update..."
-fi
-
-# Clear USE
-[ -e ${clst_make_conf} ] && sed -i -e "/^USE=\"${BINDIST} ${USE}\"/d" ${clst_make_conf}
+)
 
 export ROOT="${clst_root_path}"
 mkdir -p "$ROOT"
diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
index 88465c31..94d7c938 100755
--- a/targets/support/chroot-functions.sh
+++ b/targets/support/chroot-functions.sh
@@ -136,7 +136,7 @@ setup_emerge_opts() {
 		emerge_opts+=(--load-average "${clst_load_average}")
 	fi
 
-	if [ -n "${clst_PKGCACHE}" ] && [ -z "${clst_update_seed}" -o "${clst_update_seed}" = "no" ]
+	if [ -n "${clst_PKGCACHE}" ]
 	then
 		emerge_opts+=(--usepkg --buildpkg --binpkg-respect-use=y --newuse)
 		bootstrap_opts+=(-r)
-- 
2.26.2

Re: [gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Daniel Cordero]

On Mon, Jan 18, 2021 at 03:21:52PM -0500, Matt Turner wrote:
> There should be no need for custom updates, and if there are exceptional
> circumstances the new --enter-chroot option can be used.
> 

There is still a need, and if I'm reading these patches right,
--enter-chroot would not satisfy that need.

enter-chroot is interactive, and, depending on hardware speed, the time
between kicking off the catalyst run and when the chroot shell is
invoked may be some time for a human operator to monitor.

It also adds non-deterministic, un-reproducible behaviour to a build.
Having it in the spec file at least means it is included in a build
artifact.

Re: [gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Matt Turner]

On Tue, Jan 19, 2021 at 6:04 AM Daniel Cordero <gentoo.catalyst@xxoo.ws> wrote:
> On Mon, Jan 18, 2021 at 03:21:52PM -0500, Matt Turner wrote:
> > There should be no need for custom updates, and if there are exceptional
> > circumstances the new --enter-chroot option can be used.
> >
>
> There is still a need, and if I'm reading these patches right,
> --enter-chroot would not satisfy that need.
>
> enter-chroot is interactive, and, depending on hardware speed, the time
> between kicking off the catalyst run and when the chroot shell is
> invoked may be some time for a human operator to monitor.
>
> It also adds non-deterministic, un-reproducible behaviour to a build.
> Having it in the spec file at least means it is included in a build
> artifact.

Can you think of a case where update_seed would be needed? I tried,
and couldn't think of anything, but I might not be imaginative enough.

My thinking is that with @changed-subslots, there's essentially no
need to update the seed stage further. But there have been some
exceptional cases that don't really fit update_seed (e.g., things that
cannot be fixed with emerge ...).

Re: [gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Matt Turner]

On Tue, Jan 19, 2021 at 9:58 AM Matt Turner <mattst88@gentoo.org> wrote:
>
> On Tue, Jan 19, 2021 at 6:04 AM Daniel Cordero <gentoo.catalyst@xxoo.ws> wrote:
> > On Mon, Jan 18, 2021 at 03:21:52PM -0500, Matt Turner wrote:
> > > There should be no need for custom updates, and if there are exceptional
> > > circumstances the new --enter-chroot option can be used.
> > >
> >
> > There is still a need, and if I'm reading these patches right,
> > --enter-chroot would not satisfy that need.
> >
> > enter-chroot is interactive, and, depending on hardware speed, the time
> > between kicking off the catalyst run and when the chroot shell is
> > invoked may be some time for a human operator to monitor.
> >
> > It also adds non-deterministic, un-reproducible behaviour to a build.
> > Having it in the spec file at least means it is included in a build
> > artifact.
>
> Can you think of a case where update_seed would be needed? I tried,
> and couldn't think of anything, but I might not be imaginative enough.
>
> My thinking is that with @changed-subslots, there's essentially no
> need to update the seed stage further. But there have been some
> exceptional cases that don't really fit update_seed (e.g., things that
> cannot be fixed with emerge ...).

ping?

Re: [gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Daniel Cordero]

On Tue, Jan 19, 2021 at 09:58:15AM -0500, Matt Turner wrote:
> On Tue, Jan 19, 2021 at 6:04 AM Daniel Cordero wrote:
> > On Mon, Jan 18, 2021 at 03:21:52PM -0500, Matt Turner wrote:
> > > There should be no need for custom updates, and if there are exceptional
> > > circumstances the new --enter-chroot option can be used.
> > >
> >
> > There is still a need, and if I'm reading these patches right,
> > --enter-chroot would not satisfy that need.
> >
> > enter-chroot is interactive, and, depending on hardware speed, the time
> > between kicking off the catalyst run and when the chroot shell is
> > invoked may be some time for a human operator to monitor.
> >
> > It also adds non-deterministic, un-reproducible behaviour to a build.
> > Having it in the spec file at least means it is included in a build
> > artifact.
> 
> Can you think of a case where update_seed would be needed? I tried,
> and couldn't think of anything, but I might not be imaginative enough.
> 
> My thinking is that with @changed-subslots, there's essentially no
> need to update the seed stage further. But there have been some
> exceptional cases that don't really fit update_seed (e.g., things that
> cannot be fixed with emerge ...).

update_seed has been very useful when dealing with perl/python version
upgrades, where large swathes of packages need to be updated at the same
time. I'm not sure if @changed-subslots has in any way resolved this,
but it's convenient to have an override.

Another example was a recent bash/readline update, where the stage3 had
readline.so.5 and built the new stage1 bash against that, but installed
readline.so.6 into the stage1root, hence the stage2 build was broken.

Re: [gentoo-catalyst] [PATCH 11/11] catalyst: Remove update_seed spec option

[Matt Turner]

On Thu, Jan 21, 2021 at 5:20 AM Daniel Cordero <gentoo.catalyst@xxoo.ws> wrote:
>
> On Tue, Jan 19, 2021 at 09:58:15AM -0500, Matt Turner wrote:
> > On Tue, Jan 19, 2021 at 6:04 AM Daniel Cordero wrote:
> > > On Mon, Jan 18, 2021 at 03:21:52PM -0500, Matt Turner wrote:
> > > > There should be no need for custom updates, and if there are exceptional
> > > > circumstances the new --enter-chroot option can be used.
> > > >
> > >
> > > There is still a need, and if I'm reading these patches right,
> > > --enter-chroot would not satisfy that need.
> > >
> > > enter-chroot is interactive, and, depending on hardware speed, the time
> > > between kicking off the catalyst run and when the chroot shell is
> > > invoked may be some time for a human operator to monitor.
> > >
> > > It also adds non-deterministic, un-reproducible behaviour to a build.
> > > Having it in the spec file at least means it is included in a build
> > > artifact.
> >
> > Can you think of a case where update_seed would be needed? I tried,
> > and couldn't think of anything, but I might not be imaginative enough.
> >
> > My thinking is that with @changed-subslots, there's essentially no
> > need to update the seed stage further. But there have been some
> > exceptional cases that don't really fit update_seed (e.g., things that
> > cannot be fixed with emerge ...).
>
> update_seed has been very useful when dealing with perl/python version
> upgrades, where large swathes of packages need to be updated at the same
> time. I'm not sure if @changed-subslots has in any way resolved this,
> but it's convenient to have an override.

Okay, thanks. I'll do some testing in these areas.

> Another example was a recent bash/readline update, where the stage3 had
> readline.so.5 and built the new stage1 bash against that, but installed
> readline.so.6 into the stage1root, hence the stage2 build was broken.

This is *exactly* the situation @changed-subslots is for. Please read
the commit message in 05/11.

[gentoo-catalyst] [PATCH] Fix specifying target_profiles in repo_name:path format

[Daniel Cordero]

From: Daniel Cordero <catalyst@0xdc.io>

This fixes commit a500400592aaa309

Signed-off-by: Daniel Cordero <catalyst@0xdc.io>
---
 catalyst/base/stagebase.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index a4da7eb3..448d6265 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -814,10 +814,15 @@ class StageBase(TargetBase, ClearBase, GenBase):
         make_profile = Path(self.settings['chroot_path'] + self.settings['port_conf'],
                             'make.profile')
         make_profile.unlink(missing_ok=True)
+        try:
+            repo_name, target_profile = self.settings['target_profile'].split(":", 1)
+        except ValueError:
+            repo_name = self.settings['repo_name']
+            target_profile = self.settings['target_profile']
         make_profile.symlink_to(Path('../..' + self.settings['repo_basedir'],
-                                     self.settings['repo_name'],
+                                     repo_name,
                                      'profiles',
-                                     self.settings['target_profile']),
+                                     target_profile),
                                 target_is_directory=True)
 
     def setup_confdir(self):
-- 
2.26.2

[gentoo-catalyst] Re: [PATCH] Fix specifying target_profiles in repo_name:path format

[Matt Turner]

On Thu, Feb 25, 2021 at 8:09 AM Daniel Cordero <gentoo.catalyst@xxoo.ws> wrote:
>
> From: Daniel Cordero <catalyst@0xdc.io>
>
> This fixes commit a500400592aaa309

Felix also mentioned on IRC that he wanted this functionality.

<felixb> | mattst88_: Ok thank you for the information. I will look
into the profile problem after I'm done with the PORTDIR removal
patches. So guess the right approach would be to restore the eselect
call in some form, improving the documentation and possibly ensuring
that eselect is only called once. I think it would also make sense to
remove the manual symlink logic in config_profile_link, since that
would be redundant with eselect and is more limited (i.e. it
<felixb> | would take some work to adapt the manual symlinking to deal
with profiles outside of the main repo and parsing profile names such
as myrepo:myprofile, which eselect can resolve).

So, cc'ing him to see what he thinks.

[gentoo-catalyst] Re: [PATCH] Fix specifying target_profiles in repo_name:path format

[Felix Bier]

Am Samstag, dem 27.02.2021 um 13:52 -0500 schrieb Matt Turner:
> On Thu, Feb 25, 2021 at 8:09 AM Daniel Cordero
> <gentoo.catalyst@xxoo.ws> wrote:
> > 
> > From: Daniel Cordero <catalyst@0xdc.io>
> > 
> > This fixes commit a500400592aaa309
> 
> Felix also mentioned on IRC that he wanted this functionality.
> 
> <felixb> | mattst88_: Ok thank you for the information. I will look
> into the profile problem after I'm done with the PORTDIR removal
> patches. So guess the right approach would be to restore the eselect
> call in some form, improving the documentation and possibly ensuring
> that eselect is only called once. I think it would also make sense to
> remove the manual symlink logic in config_profile_link, since that
> would be redundant with eselect and is more limited (i.e. it
> <felixb> | would take some work to adapt the manual symlinking to
> deal
> with profiles outside of the main repo and parsing profile names such
> as myrepo:myprofile, which eselect can resolve).
> 
> So, cc'ing him to see what he thinks.

Hi, I'm happy with how Daniel solved this. I was worried about the
maintenance cost of duplicating eselect functionality, but this is very
compact.

I assumed that eselect would resolve profile names without repo: prefix
by looping over all repos to find a matching profile, but after
analysing eselect it seems such profile names are expected to always be
in the main repo. So using eselect would give no addtional
functionality compared to this patch.

I did not find any helper functions in the portage library for handling
this task.

Possibly we could improve error reporting (failing early if the link is
broken, instead of waiting for emerge to fail later), but I'm fine with
this being done in a separate step.

Best regards,
Felix