[gentoo-catalyst] [RFC PATCH] Use a 3 stage livecd build

[Daniel Cordero]

From: Daniel Cordero <catalyst@××××.io>

The current (2 stage) livecd process requires a full stage3/4 tarball
that can install genkernel and compile a kernel for the livecd. Once the
kernel is built, packages are then unmerged to reduce the size of the CD
image. This means explicity removing every unnecessary package that may
be installed from the system set, or additional stage4 world sets.

Instead, an embedded target could be used, or a stage1 target be
abused, to create minimal images for the livecd environment.

This patch moves some actions from livecd-stage1 and livecd-stage2 into
a new target: livecd-stage3.

livecd-stage1 is now very similar to a regular stage4, where additional
packages can be installed through the package manager. This step would
be optional for embedded stages without a package manager.

livecd-stage2 now uses a stage3 to build a kernel and reusable "cdtar".
The seed stage is not retained.

livecd-stage3 takes the built kernel and modules, and a
livecd-stage1/embedded/stage3/stage4 tarball. Without package manager
interaction, it combines these into a livecd. Modules for the built
kernels are extracted into the looped image.
---

The code basically copies the existing livecd-stage1/livecd-stage2 code
in the livecd-stage3 target and support scripts. I'm sure this could be
cleaned up to remove the references in livecd-stage1.py/livecd-stage2.py,
although I have not done that here.

I did also remove the module blacklisting, X11 handling and
gentoo-release-livecd/dvd file copies that was in the livecd-stage2.
They'd need to be re-added.

The livecd-stage3 specfile requires the 'profile:' keyword, even though
it is not required. This is because it is defined as a requirement by
StageBase.__init__()


 catalyst/targets/livecd_stage1.py   | 14 +++++-
 catalyst/targets/livecd_stage2.py   | 69 ++---------------------------
 catalyst/targets/livecd_stage3.py   | 61 +++++++++++++++++++++++++
 targets/livecd-stage1/controller.sh |  5 +++
 targets/livecd-stage3/controller.sh | 51 +++++++++++++++++++++
 5 files changed, 133 insertions(+), 67 deletions(-)
 create mode 100644 catalyst/targets/livecd_stage3.py
 create mode 100755 targets/livecd-stage3/controller.sh

diff --git a/catalyst/targets/livecd_stage1.py b/catalyst/targets/livecd_stage1.py
index f0b6be8b..0236eb2f 100644
--- a/catalyst/targets/livecd_stage1.py
+++ b/catalyst/targets/livecd_stage1.py
@@ -4,7 +4,6 @@ LiveCD stage1 target
 # NOTE: That^^ docstring has influence catalyst-spec(5) man page generation.
 
 from catalyst.support import normpath
-
 from catalyst.base.stagebase import StageBase
 
 
@@ -13,9 +12,14 @@ class livecd_stage1(StageBase):
     Builder class for LiveCD stage1.
     """
     required_values = frozenset([
-        "livecd/packages",
     ])
     valid_values = required_values | frozenset([
+        "livecd/empty",
+        "livecd/fsscript",
+        "livecd/packages",
+        "livecd/rm",
+        "livecd/root_overlay",
+        "livecd/unmerge",
         "livecd/use",
     ])
 
@@ -32,7 +36,13 @@ class livecd_stage1(StageBase):
             "chroot_setup",
             "setup_environment",
             "build_packages",
+            "root_overlay",
+            "fsscript",
+            "unmerge",
+            "bootloader",
             "unbind",
+            "remove",
+            "empty",
             "clean",
         ]
         self.set_completion_action_sequences()
diff --git a/catalyst/targets/livecd_stage2.py b/catalyst/targets/livecd_stage2.py
index 88c0d95c..b459288b 100644
--- a/catalyst/targets/livecd_stage2.py
+++ b/catalyst/targets/livecd_stage2.py
@@ -16,42 +16,11 @@ class livecd_stage2(StageBase):
         "boot/kernel",
     ])
     valid_values = required_values | frozenset([
-        "livecd/bootargs",
-        "livecd/cdtar",
-        "livecd/depclean",
-        "livecd/empty",
-        "livecd/fsops",
-        "livecd/fsscript",
-        "livecd/fstype",
         "livecd/gk_mainargs",
-        "livecd/iso",
         "livecd/linuxrc",
-        "livecd/modblacklist",
-        "livecd/motd",
-        "livecd/overlay",
-        "livecd/rcadd",
-        "livecd/rcdel",
-        "livecd/readme",
-        "livecd/rm",
-        "livecd/root_overlay",
-        "livecd/type",
-        "livecd/unmerge",
-        "livecd/users",
-        "livecd/verify",
-        "livecd/volid",
-        "livecd/xdm",
-        "livecd/xinitrc",
-        "livecd/xsession",
         "portage_overlay",
     ])
 
-    def __init__(self, spec, addlargs):
-        StageBase.__init__(self, spec, addlargs)
-        if "livecd/type" not in self.settings:
-            self.settings["livecd/type"] = "generic-livecd"
-
-        file_locate(self.settings, ["cdtar", "controller_file"])
-
     def set_spec_prefix(self):
         self.settings["spec_prefix"] = "livecd"
 
@@ -65,26 +34,9 @@ class livecd_stage2(StageBase):
         super(livecd_stage2, self).set_target_path()
         clear_dir(self.settings['target_path'])
 
-    def run_local(self):
-        # what modules do we want to blacklist?
-        if "livecd/modblacklist" in self.settings:
-            path = normpath(self.settings["chroot_path"] +
-                            "/etc/modprobe.d/blacklist.conf")
-            try:
-                with open(path, "a") as myf:
-                    myf.write("\n#Added by Catalyst:")
-                    # workaround until config.py is using configparser
-                    if isinstance(self.settings["livecd/modblacklist"], str):
-                        self.settings["livecd/modblacklist"] = self.settings[
-                            "livecd/modblacklist"].split()
-                    for x in self.settings["livecd/modblacklist"]:
-                        myf.write("\nblacklist "+x)
-            except:
-                self.unbind()
-                raise CatalystError("Couldn't open " +
-                                    self.settings["chroot_path"] +
-                                    "/etc/modprobe.d/blacklist.conf.",
-                                    print_traceback=True)
+    def set_stage_path(self):
+        self.set_target_path()
+        self.settings['stage_path'] = self.settings['target_path']
 
     def set_action_sequence(self):
         self.settings['action_sequence'] = [
@@ -95,24 +47,11 @@ class livecd_stage2(StageBase):
             "bind",
             "chroot_setup",
             "setup_environment",
-            "run_local",
             "build_kernel"
         ]
         if "fetch" not in self.settings["options"]:
             self.settings['action_sequence'] += [
                 "bootloader",
-                "preclean",
-                "livecd_update",
-                "root_overlay",
-                "fsscript",
-                "rcupdate",
-                "unmerge",
                 "unbind",
-                "remove",
-                "empty",
-                "clean",
-                "target_setup",
-                "setup_overlay",
-                "create_iso",
             ]
-        self.settings["action_sequence"].append("clear_autoresume")
+        self.set_completion_action_sequences()
diff --git a/catalyst/targets/livecd_stage3.py b/catalyst/targets/livecd_stage3.py
new file mode 100644
index 00000000..a5df72d3
--- /dev/null
+++ b/catalyst/targets/livecd_stage3.py
@@ -0,0 +1,61 @@
+"""
+LiveCD stage3 target
+"""
+# NOTE: That^^ docstring has influence catalyst-spec(5) man page generation.
+
+from catalyst.support import normpath
+from catalyst.fileops import clear_dir
+from catalyst.targets.livecd_stage1 import livecd_stage1
+
+
+class livecd_stage3(livecd_stage1):
+    """
+    Builder class for LiveCD stage3.
+
+    Wraps any stage into a CD image
+
+    Requires a cached kernel name and cdtar
+    """
+    required_values = frozenset([
+        "boot/kernel",
+        "livecd/cdtar",
+    ])
+    valid_values = required_values | frozenset([
+        "livecd/bootargs",
+        "livecd/empty",
+        "livecd/fsscript",
+        "livecd/fstype",
+        "livecd/iso",
+        "livecd/rm",
+        "livecd/root_overlay",
+        "livecd/type",
+        "livecd/verify",
+        "livecd/volid",
+    ])
+
+    def set_action_sequence(self):
+        self.settings['action_sequence'] = [
+            "unpack",
+            "bind",
+            "setup_environment",
+            "root_overlay",
+            "fsscript",
+            "bootloader",
+            "unbind",
+            "remove",
+            "empty",
+            "target_setup",
+            "create_iso",
+            "clear_autoresume",
+            "remove_chroot",
+        ]
+
+    def set_target_path(self):
+        '''Set the target path for the finished stage.
+
+        This method runs the StageBase.set_target_path method,
+        and additionally creates a staging directory for assembling
+        the final components needed to produce the iso image.
+        '''
+        super(livecd_stage1, self).set_target_path()
+        clear_dir(self.settings['target_path'])
diff --git a/targets/livecd-stage1/controller.sh b/targets/livecd-stage1/controller.sh
index ae897da9..0d472f1f 100755
--- a/targets/livecd-stage1/controller.sh
+++ b/targets/livecd-stage1/controller.sh
@@ -12,5 +12,10 @@ case $1 in
 		exec_in_chroot \
 			${clst_shdir}/${clst_target}/chroot.sh
 		;;
+
+	fsscript)
+		exec_in_chroot ${clst_fsscript}
+		;;
+
 esac
 exit $?
diff --git a/targets/livecd-stage3/controller.sh b/targets/livecd-stage3/controller.sh
new file mode 100755
index 00000000..632e8ccd
--- /dev/null
+++ b/targets/livecd-stage3/controller.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+source ${clst_shdir}/support/functions.sh
+
+## START RUNSCRIPT
+
+case $1 in
+	fsscript)
+		exec_in_chroot ${clst_fsscript}
+		;;
+
+	bootloader)
+		shift
+		# Here is where we poke in our identifier
+		touch $1/livecd
+
+		# We create a firmware directory, if necessary
+		if [ -e ${clst_chroot_path}/lib/firmware.tar.bz2 ]
+		then
+			echo "Creating firmware directory in $1"
+			mkdir -p $1/firmware
+			# TODO: Unpack firmware into $1/firmware and remove it from the
+			# chroot so newer livecd-tools/genkernel can find it and unpack it.
+		fi
+
+		# Move over the readme (if applicable)
+		if [ -n "${clst_livecd_readme}" ]
+		then
+			cp -f ${clst_livecd_readme} $1/README.txt
+		else
+			cp -f ${clst_sharedir}/livecd/files/README.txt $1
+		fi
+
+		extract_cdtar $1
+
+		for x in ${clst_boot_kernel}; do
+			extract_modules ${clst_chroot_path} $x
+		done
+		;;
+
+	target_image_setup)
+		shift
+		${clst_shdir}/support/target_image_setup.sh $1
+		;;
+
+	iso)
+		shift
+		${clst_shdir}/support/create-iso.sh $1
+		;;
+esac
+exit $?
-- 
2.26.2