From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id C6C341382C5 for ; Wed, 20 May 2020 03:43:23 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 16D13E0922; Wed, 20 May 2020 03:43:23 +0000 (UTC) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 021C4E0922 for ; Wed, 20 May 2020 03:43:22 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id 145so894891pfw.13 for ; Tue, 19 May 2020 20:43:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iMjdNpnHHg/oHi6YFydX4ENJu51ZBDBYYoQFTrNN74c=; b=iC6QIT0eY9xOrjOnkCVHwnMLTpx24cGzpIPMeKxgmED1RidmtZtp63Sc7/RIbLBXS1 bIoD+JmmWHLZ+2HsIw5gBFSIR067tTEJlvuaX9FhGNdyrJhgwKsLQQaOAr7YMVwUSqhp Ecn6k3Gj0MB6wMTkdhU7Dbea8Hc0tz/B/BY3Zs+Io+tWGtdTketXouoACHeJscXdqy7V 2t7cXL4v0vgaKowpFH5vLXEeBpexy7aXC/7w0seEm+bPXoRaAnONV0q3DIoRA7BxbYLd Iin4R+5KNgunFWaHG23WVCy5p2kOkE64Ggu05MsEOIyQV5GxKu73O3rs7yY1+JIOWhPd nvig== X-Gm-Message-State: AOAM531RGJzztZy1xjLQFV0p7efEEKRISbNKmeAyVlSt61VXFGonTzNe QA7HI4J3W6U6cbI9Xenb++6TzG3M X-Google-Smtp-Source: ABdhPJw+PnuNi4YaatPrz8PNQqEwvAj6yOiw32rIyh00mE2OClDwtqZDesSBYPIPwydYTxdBoPgE/A== X-Received: by 2002:aa7:9543:: with SMTP id w3mr2273328pfq.191.1589946201491; Tue, 19 May 2020 20:43:21 -0700 (PDT) Received: from localhost ([134.134.137.77]) by smtp.gmail.com with ESMTPSA id p4sm740978pff.159.2020.05.19.20.43.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 May 2020 20:43:20 -0700 (PDT) From: Matt Turner To: gentoo-catalyst@lists.gentoo.org Cc: Matt Turner Subject: [gentoo-catalyst] [PATCH 19/21] catalyst: Set jobs/load-average via catalyst.conf Date: Tue, 19 May 2020 20:42:24 -0700 Message-Id: <20200520034226.2870937-19-mattst88@gentoo.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200520034226.2870937-1-mattst88@gentoo.org> References: <20200520034226.2870937-1-mattst88@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-catalyst@lists.gentoo.org Reply-to: gentoo-catalyst@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Archives-Salt: 296fed64-743b-49d0-bb89-525fecd24b16 X-Archives-Hash: 5c294634fc849c72a7165d35cd03fe1e We currently have two mechanisms of setting MAKEOPTS: in spec files and in catalystrc. Setting makeopts in spec files doesn't make sense. The spec should describe the thing that's being built and not contain options that are specific to the build system. Setting makeopts via catalystrc is better, but it only applies to the actual build system invocations, leaving emerge to run jobs serially or again requiring configuration specific to the build machine to be put into the spec file. For example: update_seed_command: ... --jobs 5 --load-average 5 With jobs and load-average specified in catalyst.conf, catalyst has the information required to configure both emerge and the build systems emerge executes. This removes the undocumented makeopts spec file option and replaces it with jobs and load-average settings in catalyst.conf. Signed-off-by: Matt Turner --- catalyst/base/stagebase.py | 12 +++++------- catalyst/defaults.py | 2 ++ doc/catalyst-config.5.txt | 15 ++++++++++++--- etc/catalyst.conf | 8 ++++++++ etc/catalystrc | 3 --- targets/support/chroot-functions.sh | 8 ++++++++ 6 files changed, 35 insertions(+), 13 deletions(-) diff --git a/catalyst/base/stagebase.py b/catalyst/base/stagebase.py index 5a8cd1df..bc721ad4 100644 --- a/catalyst/base/stagebase.py +++ b/catalyst/base/stagebase.py @@ -56,7 +56,6 @@ class StageBase(TargetBase, ClearBase, GenBase): "hostuse", "kerncache_path", "ldflags", - "makeopts", "pkgcache_path", "portage_confdir", "portage_overlay", @@ -1290,12 +1289,11 @@ class StageBase(TargetBase, ClearBase, GenBase): continue log.warning("Not making envar for '%s', is a dict", x) - if "makeopts" in self.settings: - if isinstance(self.settings["makeopts"], str): - self.env["MAKEOPTS"] = self.settings["makeopts"] - else: - # ensure makeopts is a string - self.env["MAKEOPTS"] = ' '.join(self.settings["makeopts"]) + makeopts = [] + for flag, setting in {'j': 'jobs', 'l': 'load-average'}.items(): + if setting in self.settings: + makeopts.append(f'-{flag}{self.settings[setting]}') + self.env['MAKEOPTS'] = ' '.join(makeopts) log.debug('setup_environment(); env = %r', self.env) diff --git a/catalyst/defaults.py b/catalyst/defaults.py index 14f671fe..b31d5b50 100644 --- a/catalyst/defaults.py +++ b/catalyst/defaults.py @@ -12,6 +12,8 @@ valid_config_file_values = frozenset([ "digests", "distdir", "envscript", + "jobs", + "load-average", "options", "port_logdir", "repo_basedir", diff --git a/doc/catalyst-config.5.txt b/doc/catalyst-config.5.txt index 7ac9a2a3..cbef6092 100644 --- a/doc/catalyst-config.5.txt +++ b/doc/catalyst-config.5.txt @@ -48,9 +48,9 @@ $ python3 -c 'import hashlib; print(hashlib.algorithms_available)' *envscript*:: Environment script location, which allows users to set options such as -HTTP proxies, `MAKEOPTS`, `GENTOO_MIRRORS`, or any other environment -variables needed for building. The envscript file sets environment -variables using POSIX shell notation: +HTTP proxies, `GENTOO_MIRRORS`, or any other environment variables +needed for building. The envscript file sets environment variables +using POSIX shell notation: + --------------------------------- export FOO="bar" @@ -136,6 +136,15 @@ written to the target's make.conf if it is not the default value of Other settings ~~~~~~~~~~~~~~ +*jobs*:: +Integral value passed to *emerge(1)* as the parameter to --jobs and is +used to define *MAKEOPTS* during the target build. + +*load-average*:: +Floating-point value passed to *emerge(1)* as the parameter to +--load-average and is used to define *MAKEOPTS* during the target +build. + *sharedir*:: Catalyst runtime script location. `/usr/share/catalyst` should work for most default installations. If you are running catalyst from a Git diff --git a/etc/catalyst.conf b/etc/catalyst.conf index 2272cb86..81693c25 100644 --- a/etc/catalyst.conf +++ b/etc/catalyst.conf @@ -76,3 +76,11 @@ options = [ # WARNING: If you use too much RAM everything will fail horribly and it is not our fault. # set size of /var/tmp/portage tmpfs in gigabytes # var_tmpfs_portage = 16 + +# Integral value passed to emerge as the parameter to --jobs and is used to +# define MAKEOPTS during the target build. +# jobs = 4 + +# Floating-point value passed to emerge as the parameter to --load-average and +# is used to define MAKEOPTS during the target build. +# load-average = 4.0 diff --git a/etc/catalystrc b/etc/catalystrc index bcd729af..e7904128 100755 --- a/etc/catalystrc +++ b/etc/catalystrc @@ -1,5 +1,2 @@ #!/bin/bash # This is an example catalystrc. As such, it doesn't actually *do* anything. - -# Uncomment the following to increase the number of threads used to compile. -# export MAKEOPTS="-j16" diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh index b531eb6a..4005a1d8 100755 --- a/targets/support/chroot-functions.sh +++ b/targets/support/chroot-functions.sh @@ -130,6 +130,14 @@ setup_emerge_opts() { emerge_opts+=(--fetchonly) bootstrap_opts+=(-f) fi + if [ -n "${clst_jobs}" ] + then + emerge_opts+=(--jobs "${clst_jobs}") + fi + if [ -n "${clst_load_average}" ] + then + emerge_opts+=(--load-average "${clst_load_average}") + fi if [ -n "${clst_PKGCACHE}" ] && [ -z "${clst_update_seed}" -o "${clst_update_seed}" = "no" ] then -- 2.26.2