[gentoo-commits] proj/portage:master commit in: man/, lib/_emerge/, cnf/

[Mike Gilbert]

commit:     353c61912b134b326da5ac16ef1d4bc74b8967d1
Author:     KARBOWSKI Piotr <slashbeast <AT> gentoo <DOT> org>
AuthorDate: Mon Aug  1 20:55:41 2022 +0000
Commit:     Mike Gilbert <floppym <AT> gentoo <DOT> org>
CommitDate: Wed Aug  3 15:57:38 2022 +0000
URL:        https://gitweb.gentoo.org/proj/portage.git/commit/?id=353c6191

Scheduling policy switching

Adds ability to control the scheduler policy that is used for emerge and
all child processes. Mainly to interface the ability to switch to
SCHED_IDLE as the solution to keep interactive tasks unaffected by
building process happening in the background.

On a test sample N=1 with AMD Ryzen 5950x and 64 GB of ram building
sys-devel/gcc with lto enabled significantly reduces responsiveness of
the system, even with CONFIG_SCHED_AUTOGROUP and PREEMPT enabled. Using
a web browser result in visible lags, video playback in web browser,
when using CPU decoding, also suffers greatly.

Switching Portage to SCHED_IDLE (PORTAGE_SCHEDULING_POLICY="idle")
results in no visible slowdowns and responsiveness is as if nothing in
the background was happening.

This is especially worthy feature when running on powerful CPUs, where
users often opt in to build not only with parallel build jobs, but also
with multiple packages at once. Anyone running with PORTAGE_NICENESS="19" will
undoubtedly want to use this feature to force SCHED_IDLE policy.

Closes: https://github.com/gentoo/portage/pull/861
Signed-off-by: KARBOWSKI Piotr <slashbeast <AT> gentoo.org>
Signed-off-by: Mike Gilbert <floppym <AT> gentoo.org>

 cnf/make.conf.example  | 17 +++++++++++++++++
 lib/_emerge/actions.py | 45 +++++++++++++++++++++++++++++++++++++++++++++
 man/make.conf.5        | 16 ++++++++++++++++
 3 files changed, 78 insertions(+)

diff --git a/cnf/make.conf.example b/cnf/make.conf.example
index 5b2229465..2e33a6e50 100644
--- a/cnf/make.conf.example
+++ b/cnf/make.conf.example
@@ -291,6 +291,23 @@
 #     unset.
 #PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
 #
+# PORTAGE_SCHEDULING_POLICY allows changing the current scheduling policy. The
+# supported options are 'other', 'batch', 'idle', 'fifo' and 'round-robin'. When
+# unset, the scheduling policy remains unchanged, by default Linux uses 'other'
+# policy. Users that wish to minimize the Portage's impact on system
+# responsiveness should set scheduling policy to 'idle' which significantly
+# reduces the disruption to the rest of the system by scheduling Portage as
+# extremely low priority processes.
+#
+#PORTAGE_SCHEDULING_POLICY="idle"
+#
+# PORTAGE_SCHEDULING_PRIORITY allows changing the priority (1-99) of the current
+# scheduling policy, only applies if PORTAGE_SCHEDULING_POLICY is set to 'fifo'
+# or 'round-robin', for others the only supported priority is 0, If unset,
+# defaults to lowest priority of the selected scheduling policy.
+#
+#PORTAGE_SCHEDULING_PRIORITY="99"
+#
 # AUTOCLEAN enables portage to automatically clean out older or overlapping
 #     packages from the system after every successful merge. This is the
 #     same as running 'emerge -c' after every merge. Set with: "yes" or "no".

diff --git a/lib/_emerge/actions.py b/lib/_emerge/actions.py
index e2f3f2ccf..e79bb30c0 100644
--- a/lib/_emerge/actions.py
+++ b/lib/_emerge/actions.py
@@ -3054,6 +3054,7 @@ def config_protect_check(trees):
 def apply_priorities(settings):
     ionice(settings)
     nice(settings)
+    set_scheduling_policy(settings)
 
 
 def nice(settings):
@@ -3094,6 +3095,50 @@ def ionice(settings):
         )
 
 
+def set_scheduling_policy(settings):
+    scheduling_policy = settings.get("PORTAGE_SCHEDULING_POLICY")
+    scheduling_priority = settings.get("PORTAGE_SCHEDULING_PRIORITY")
+
+    if platform.system() != "Linux" or not scheduling_policy:
+        return os.EX_OK
+
+    policies = {
+        "other": os.SCHED_OTHER,
+        "batch": os.SCHED_BATCH,
+        "idle": os.SCHED_IDLE,
+        "fifo": os.SCHED_FIFO,
+        "round-robin": os.SCHED_RR,
+    }
+
+    out = portage.output.EOutput()
+
+    if scheduling_policy in policies:
+        policy = policies[scheduling_policy]
+    else:
+        out.eerror("Invalid policy in PORTAGE_SCHEDULING_POLICY.")
+        out.eerror(
+            "See the make.conf(5) man page for PORTAGE_SCHEDULING_POLICY usage instructions."
+        )
+        return os.EX_USAGE
+
+    if not scheduling_priority:
+        scheduling_priority = os.sched_get_priority_min(policy)
+    else:
+        scheduling_priority = int(scheduling_priority)
+        if scheduling_priority not in range(
+            os.sched_get_priority_min(policy), os.sched_get_priority_max(policy) + 1
+        ):
+            out.eerror("Invalid priority in PORTAGE_SCHEDULING_PRIORITY.")
+            out.eerror(
+                "See the make.conf(5) man page for PORTAGE_SCHEDULING_PRIORITY usage instructions."
+            )
+            return os.EX_USAGE
+
+    os.sched_setscheduler(portage.getpid(), policy, os.sched_param(scheduling_priority))
+
+    return os.EX_OK
+
+
 def setconfig_fallback(root_config):
     setconfig = root_config.setconfig
     setconfig._create_default_config()

diff --git a/man/make.conf.5 b/man/make.conf.5
index bde92af1a..a527a3f74 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -1080,6 +1080,22 @@ will set idle io priority. For more information about ionice, see
 Portage will also set the autogroup-nice value (see fBsched\fR(7))), if
 FEATURES="pid\-sandbox" is enabled.
 .TP
+\fBPORTAGE_SCHEDULING_POLICY\fR = \fI[policy name]\fR
+Allows changing the current scheduling policy. The supported options are
+\fBother\fR, \fBbatch\fR, \fBidle\fR, \fBfifo\fR, and \fBround-robin\fR. When
+unset, the scheduling policy remains unchanged, by default Linux uses 'other'
+policy. Users that wish to minimize the Portage's impact on system
+responsiveness should set scheduling policy to \fBidle\fR, which significantly
+reduces the disruption to the rest of the system by scheduling Portage as
+extremely low priority processes. see \fBsched\fR(7) for more information.
+.TP
+\fBPORTAGE_SCHEDULING_PRIORITY\fR = \fI[priority]\fR
+Allows changing the priority (1-99) of the current scheduling policy, only
+applies if PORTAGE _SCHEDULING_POLICY is set to 'fifo' or 'round-robin',
+for others the only supported priority is 0, If unset, defaults to lowest
+priority of the selected scheduling policy. For more information about
+scheduler, see \fBsched\fR(7). This variable is unset by default.
+.TP
 .B PORTAGE_LOG_FILTER_FILE_CMD
 This variable specifies a command that filters build log output to a
 log file. In order to filter ANSI escape codes from build logs,