From: Matt Turner <mattst88@gentoo.org>
To: gentoo-catalyst@lists.gentoo.org
Cc: Matt Turner <mattst88@gentoo.org>
Subject: [gentoo-catalyst] [PATCH 19/21] catalyst: Set jobs/load-average via catalyst.conf
Date: Tue, 19 May 2020 20:42:24 -0700 [thread overview]
Message-ID: <20200520034226.2870937-19-mattst88@gentoo.org> (raw)
In-Reply-To: <20200520034226.2870937-1-mattst88@gentoo.org>
We currently have two mechanisms of setting MAKEOPTS: in spec files and
in catalystrc.
Setting makeopts in spec files doesn't make sense. The spec should
describe the thing that's being built and not contain options that are
specific to the build system.
Setting makeopts via catalystrc is better, but it only applies to the
actual build system invocations, leaving emerge to run jobs serially or
again requiring configuration specific to the build machine to be put
into the spec file. For example:
update_seed_command: ... --jobs 5 --load-average 5
With jobs and load-average specified in catalyst.conf, catalyst has the
information required to configure both emerge and the build systems
emerge executes.
This removes the undocumented makeopts spec file option and replaces it
with jobs and load-average settings in catalyst.conf.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
---
catalyst/base/stagebase.py | 12 +++++-------
catalyst/defaults.py | 2 ++
doc/catalyst-config.5.txt | 15 ++++++++++++---
etc/catalyst.conf | 8 ++++++++
etc/catalystrc | 3 ---
targets/support/chroot-functions.sh | 8 ++++++++
6 files changed, 35 insertions(+), 13 deletions(-)
diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py
index 5a8cd1df..bc721ad4 100644
--- a/catalyst/base/stagebase.py
+++ b/catalyst/base/stagebase.py
@@ -56,7 +56,6 @@ class StageBase(TargetBase, ClearBase, GenBase):
"hostuse",
"kerncache_path",
"ldflags",
- "makeopts",
"pkgcache_path",
"portage_confdir",
"portage_overlay",
@@ -1290,12 +1289,11 @@ class StageBase(TargetBase, ClearBase, GenBase):
continue
log.warning("Not making envar for '%s', is a dict", x)
- if "makeopts" in self.settings:
- if isinstance(self.settings["makeopts"], str):
- self.env["MAKEOPTS"] = self.settings["makeopts"]
- else:
- # ensure makeopts is a string
- self.env["MAKEOPTS"] = ' '.join(self.settings["makeopts"])
+ makeopts = []
+ for flag, setting in {'j': 'jobs', 'l': 'load-average'}.items():
+ if setting in self.settings:
+ makeopts.append(f'-{flag}{self.settings[setting]}')
+ self.env['MAKEOPTS'] = ' '.join(makeopts)
log.debug('setup_environment(); env = %r', self.env)
diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 14f671fe..b31d5b50 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -12,6 +12,8 @@ valid_config_file_values = frozenset([
"digests",
"distdir",
"envscript",
+ "jobs",
+ "load-average",
"options",
"port_logdir",
"repo_basedir",
diff --git a/doc/catalyst-config.5.txt b/doc/catalyst-config.5.txt
index 7ac9a2a3..cbef6092 100644
--- a/doc/catalyst-config.5.txt
+++ b/doc/catalyst-config.5.txt
@@ -48,9 +48,9 @@ $ python3 -c 'import hashlib; print(hashlib.algorithms_available)'
*envscript*::
Environment script location, which allows users to set options such as
-HTTP proxies, `MAKEOPTS`, `GENTOO_MIRRORS`, or any other environment
-variables needed for building. The envscript file sets environment
-variables using POSIX shell notation:
+HTTP proxies, `GENTOO_MIRRORS`, or any other environment variables
+needed for building. The envscript file sets environment variables
+using POSIX shell notation:
+
---------------------------------
export FOO="bar"
@@ -136,6 +136,15 @@ written to the target's make.conf if it is not the default value of
Other settings
~~~~~~~~~~~~~~
+*jobs*::
+Integral value passed to *emerge(1)* as the parameter to --jobs and is
+used to define *MAKEOPTS* during the target build.
+
+*load-average*::
+Floating-point value passed to *emerge(1)* as the parameter to
+--load-average and is used to define *MAKEOPTS* during the target
+build.
+
*sharedir*::
Catalyst runtime script location. `/usr/share/catalyst` should work for
most default installations. If you are running catalyst from a Git
diff --git a/etc/catalyst.conf b/etc/catalyst.conf
index 2272cb86..81693c25 100644
--- a/etc/catalyst.conf
+++ b/etc/catalyst.conf
@@ -76,3 +76,11 @@ options = [
# WARNING: If you use too much RAM everything will fail horribly and it is not our fault.
# set size of /var/tmp/portage tmpfs in gigabytes
# var_tmpfs_portage = 16
+
+# Integral value passed to emerge as the parameter to --jobs and is used to
+# define MAKEOPTS during the target build.
+# jobs = 4
+
+# Floating-point value passed to emerge as the parameter to --load-average and
+# is used to define MAKEOPTS during the target build.
+# load-average = 4.0
diff --git a/etc/catalystrc b/etc/catalystrc
index bcd729af..e7904128 100755
--- a/etc/catalystrc
+++ b/etc/catalystrc
@@ -1,5 +1,2 @@
#!/bin/bash
# This is an example catalystrc. As such, it doesn't actually *do* anything.
-
-# Uncomment the following to increase the number of threads used to compile.
-# export MAKEOPTS="-j16"
diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
index b531eb6a..4005a1d8 100755
--- a/targets/support/chroot-functions.sh
+++ b/targets/support/chroot-functions.sh
@@ -130,6 +130,14 @@ setup_emerge_opts() {
emerge_opts+=(--fetchonly)
bootstrap_opts+=(-f)
fi
+ if [ -n "${clst_jobs}" ]
+ then
+ emerge_opts+=(--jobs "${clst_jobs}")
+ fi
+ if [ -n "${clst_load_average}" ]
+ then
+ emerge_opts+=(--load-average "${clst_load_average}")
+ fi
if [ -n "${clst_PKGCACHE}" ] && [ -z "${clst_update_seed}" -o "${clst_update_seed}" = "no" ]
then
--
2.26.2
next prev parent reply other threads:[~2020-05-20 3:43 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-20 3:42 [gentoo-catalyst] [PATCH 01/21] targets: Remove copy_{file,symlink,lib} functions Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 02/21] targets: Remove check_genkernel_version() function Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 03/21] catalyst: Drop --cli option Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 04/21] catalyst: Remove PythonDir setting Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 05/21] catalyst: Switch internal snapshot option parsing to SpecParser Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 06/21] catalyst: Remove unused decompression_mode spec option Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 07/21] catalyst: Delete unused {required,valid}_build_targets Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 08/21] catalyst: Disallow DEBUG and VERBOSE in spec files Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 09/21] catalyst: Add decompressor_search_order as valid spec option Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 10/21] catalyst: Remove spec file options from valid_config_file_values Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 11/21] catalyst: Add a missing options to valid_config_file_values Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 12/21] catalyst: Disallow config file options in spec files Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 13/21] catalyst: Remove support for source_matching="loose" Matt Turner
2020-05-20 22:47 ` Brian Dolbec
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 14/21] catalyst: Convert catalyst.conf to TOML Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 15/21] catalyst: Verify config options against valid_config_file_values Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 16/21] catalyst: Don't even try to make envars from dicts Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 17/21] catalyst: Only emit true boolean envars Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 18/21] catalyst: Support emitting int/float envars Matt Turner
2020-05-20 3:42 ` Matt Turner [this message]
[not found] ` <94424974-cfe8-9766-8712-ae6fa8bbf825@veremit.xyz>
2020-05-21 0:18 ` [gentoo-catalyst] [PATCH 19/21] catalyst: Set jobs/load-average via catalyst.conf Matt Turner
[not found] ` <9d529c77-14b3-cb7b-f2ee-32b4bbc2f464@veremit.xyz>
2020-05-21 1:19 ` Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 20/21] catalyst: Configure distcc_hosts in the config file Matt Turner
2020-05-20 3:42 ` [gentoo-catalyst] [PATCH 21/21] catalyst: Drop ConfigParser Matt Turner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200520034226.2870937-19-mattst88@gentoo.org \
--to=mattst88@gentoo.org \
--cc=gentoo-catalyst@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox