[gentoo-commits] proj/catalyst:wip/mattst88 commit in: targets/support/, catalyst/base/, etc/, catalyst/, doc/

[Matt Turner]

commit:     3269cfc50ec2eeccbb1c5aebc89fd4b38ce1b3e8
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Thu May 14 02:37:11 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed Oct 21 01:07:29 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=3269cfc5

catalyst: Set jobs/load-average via catalyst.conf

We currently have two mechanisms of setting MAKEOPTS: in spec files and
in catalystrc.

Setting makeopts in spec files doesn't make sense. The spec should
describe the thing that's being built and not contain options that are
specific to the build system.

Setting makeopts via catalystrc is better, but it only applies to the
actual build system invocations, leaving emerge to run jobs serially or
again requiring configuration specific to the build machine to be put
into the spec file. For example:

	update_seed_command: ... --jobs 5 --load-average 5

With jobs and load-average specified in catalyst.conf, catalyst has the
information required to configure both emerge and the build systems
emerge executes.

This removes the undocumented makeopts spec file option and replaces it
with jobs and load-average settings in catalyst.conf.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/base/stagebase.py          | 12 +++++-------
 catalyst/defaults.py                |  2 ++
 doc/catalyst-config.5.txt           | 15 ++++++++++++---
 etc/catalyst.conf                   |  8 ++++++++
 etc/catalystrc                      |  3 ---
 targets/support/chroot-functions.sh |  8 ++++++++
 6 files changed, 35 insertions(+), 13 deletions(-)

diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 99d638d8..afea5776 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -59,7 +59,6 @@ class StageBase(TargetBase, ClearBase, GenBase):
             "interpreter",
             "kerncache_path",
             "ldflags",
-            "makeopts",
             "pkgcache_path",
             "portage_confdir",
             "portage_overlay",
@@ -1336,12 +1335,11 @@ class StageBase(TargetBase, ClearBase, GenBase):
                     continue
                 log.warning("Not making envar for '%s', is a dict", x)
 
-        if "makeopts" in self.settings:
-            if isinstance(self.settings["makeopts"], str):
-                self.env["MAKEOPTS"] = self.settings["makeopts"]
-            else:
-                # ensure makeopts is a string
-                self.env["MAKEOPTS"] = ' '.join(self.settings["makeopts"])
+        makeopts = []
+        for flag, setting in {'j': 'jobs', 'l': 'load-average'}.items():
+            if setting in self.settings:
+                makeopts.append(f'-{flag}{self.settings[setting]}')
+        self.env['MAKEOPTS'] = ' '.join(makeopts)
 
         log.debug('setup_environment(); env = %r', self.env)
 

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index c153fcc4..7a2fe3f3 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -13,6 +13,8 @@ valid_config_file_values = frozenset([
     "digests",
     "distdir",
     "envscript",
+    "jobs",
+    "load-average",
     "options",
     "port_logdir",
     "repo_basedir",

diff --git a/doc/catalyst-config.5.txt b/doc/catalyst-config.5.txt
index 7ac9a2a3..cbef6092 100644
--- a/doc/catalyst-config.5.txt
+++ b/doc/catalyst-config.5.txt
@@ -48,9 +48,9 @@ $ python3 -c 'import hashlib; print(hashlib.algorithms_available)'
 
 *envscript*::
 Environment script location, which allows users to set options such as
-HTTP proxies, `MAKEOPTS`, `GENTOO_MIRRORS`, or any other environment
-variables needed for building.  The envscript file sets environment
-variables using POSIX shell notation:
+HTTP proxies, `GENTOO_MIRRORS`, or any other environment variables
+needed for building.  The envscript file sets environment variables
+using POSIX shell notation:
 +
 ---------------------------------
 export FOO="bar"
@@ -136,6 +136,15 @@ written to the target's make.conf if it is not the default value of
 Other settings
 ~~~~~~~~~~~~~~
 
+*jobs*::
+Integral value passed to *emerge(1)* as the parameter to --jobs and is
+used to define *MAKEOPTS* during the target build.
+
+*load-average*::
+Floating-point value passed to *emerge(1)* as the parameter to
+--load-average and is used to define *MAKEOPTS* during the target
+build.
+
 *sharedir*::
 Catalyst runtime script location.  `/usr/share/catalyst` should work for
 most default installations.  If you are running catalyst from a Git

diff --git a/etc/catalyst.conf b/etc/catalyst.conf
index 2272cb86..81693c25 100644
--- a/etc/catalyst.conf
+++ b/etc/catalyst.conf
@@ -76,3 +76,11 @@ options = [
 # WARNING: If you use too much RAM everything will fail horribly and it is not our fault.
 # set size of /var/tmp/portage tmpfs in gigabytes
 # var_tmpfs_portage = 16
+
+# Integral value passed to emerge as the parameter to --jobs and is used to
+# define MAKEOPTS during the target build.
+# jobs = 4
+
+# Floating-point value passed to emerge as the parameter to --load-average and
+# is used to define MAKEOPTS during the target build.
+# load-average = 4.0

diff --git a/etc/catalystrc b/etc/catalystrc
index bcd729af..e7904128 100755
--- a/etc/catalystrc
+++ b/etc/catalystrc
@@ -1,5 +1,2 @@
 #!/bin/bash
 # This is an example catalystrc. As such, it doesn't actually *do* anything.
-
-# Uncomment the following to increase the number of threads used to compile.
-# export MAKEOPTS="-j16"

diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
index 7222892e..89a313f7 100755
--- a/targets/support/chroot-functions.sh
+++ b/targets/support/chroot-functions.sh
@@ -130,6 +130,14 @@ setup_emerge_opts() {
 		emerge_opts+=(--fetchonly)
 		bootstrap_opts+=(-f)
 	fi
+	if [ -n "${clst_jobs}" ]
+	then
+		emerge_opts+=(--jobs "${clst_jobs}")
+	fi
+	if [ -n "${clst_load_average}" ]
+	then
+		emerge_opts+=(--load-average "${clst_load_average}")
+	fi
 
 	if [ -n "${clst_PKGCACHE}" ] && [ -z "${clst_update_seed}" -o "${clst_update_seed}" = "no" ]
 	then