Gentoo Archives: gentoo-commits

From: Zac Medico <zmedico@g.o>
To: gentoo-commits@l.g.o
Subject: [gentoo-commits] proj/portage:master commit in: man/, lib/_emerge/
Date: Mon, 14 Jun 2021 21:24:04
Message-Id: 1623705606.055abe523c2c3f6c8f1dccfb53565209222f90c1.zmedico@gentoo
1 commit: 055abe523c2c3f6c8f1dccfb53565209222f90c1
2 Author: Florian Schmaus <flo <AT> geekplace <DOT> eu>
3 AuthorDate: Sun Mar 21 11:07:38 2021 +0000
4 Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
5 CommitDate: Mon Jun 14 21:20:06 2021 +0000
6 URL: https://gitweb.gentoo.org/proj/portage.git/commit/?id=055abe52
7
8 PORTAGE_NICENESS: Consider autogroup scheduling
9
10 With Linux's autogroup scheduling feature (CONFIG_SCHED_AUTOGROUP)
11 setting a nice value on a per-process base has only an effect for
12 scheduling decisions relative to the other threads in the same
13 session (typically: the same terminal window). See the section "The
14 nice value and group scheduling" in the sched(7) man page.
15
16 Basically this means that portage "just" setting the nice value, has
17 no effect in presence of autogroup scheduling being active (which is
18 probably true for most (desktop) user systems).
19
20 This commit changes emerge to set the autogroup's nice value, instead
21 of the processes' nice value, in case autogroups are present (detected
22 by the existence of /proc/self/autogroup). The tricky part about
23 autogroup nice values is that we want restore the orignal nice value
24 once we are finished. As otherwise, the session, e.g. your terminal,
25 would continue using this value, and so would subsequently executed
26 processes. For that we use Python's atexit functinaly, to register a
27 function that will restore the orignal nice value of the autogroup.
28
29 Users may have set PORTAGE_NICENESS to a value outside of the range
30 of valid nice values [-20, 19]. Calling os.nice() with such a value
31 will simply cap the process's nice value, but writing this invalid
32 value to the autogoup pseudo-file will fail with "Invalid argument".
33 Since os.nice() returns the current nice value, we simply use the
34 returned value to set the autogroup nice value.
35
36 Portage would previously always change the nice value to zero, even if
37 the user did not explicitly request so. Now we do not change the nice
38 value unless requested.
39
40 Closes: https://github.com/gentoo/portage/pull/727
41 Bug: https://bugs.gentoo.org/777492
42 Signed-off-by: Florian Schmaus <flo <AT> geekplace.eu>
43 Signed-off-by: Zac Medico <zmedico <AT> gentoo.org>
44
45 lib/_emerge/actions.py | 48 +++++++++++++++++++++++++++++++++++++++++++++---
46 man/make.conf.5 | 10 +++++++++-
47 2 files changed, 54 insertions(+), 4 deletions(-)
48
49 diff --git a/lib/_emerge/actions.py b/lib/_emerge/actions.py
50 index 1946f49df..df5c86c6c 100644
51 --- a/lib/_emerge/actions.py
52 +++ b/lib/_emerge/actions.py
53 @@ -14,6 +14,7 @@ import textwrap
54 import time
55 import warnings
56 from itertools import chain
57 +from pathlib import Path
58
59 import portage
60 portage.proxy.lazyimport.lazyimport(globals(),
61 @@ -2634,14 +2635,55 @@ def apply_priorities(settings):
62 nice(settings)
63
64 def nice(settings):
65 + nice_value: str = settings.get("PORTAGE_NICENESS", "").strip()
66 + if not nice_value:
67 + return
68 +
69 try:
70 - os.nice(int(settings.get("PORTAGE_NICENESS", "0")))
71 + current_nice_value = os.nice(int(nice_value))
72 + # Calling os.nice() with a value outside of the valid range of
73 + # nice values, e.g. 20, caps the process's nice value. This is
74 + # because the argument of os.nice() is not an absolute value,
75 + # but the increment to the process's current nice
76 + # value. Hence users may use PORTAGE_NICENESS=20 without any
77 + # issues here. However, below we write nice_value potentially
78 + # to /proc/self/autogroup, which will only accept valid nice
79 + # values. Therefore we simply set nice_value to what os.nice()
80 + # returned (i.e. the process's current nice value).
81 + nice_value = str(current_nice_value)
82 except (OSError, ValueError) as e:
83 out = portage.output.EOutput()
84 - out.eerror("Failed to change nice value to '%s'" % \
85 - settings.get("PORTAGE_NICENESS", "0"))
86 + out.eerror(f"Failed to change nice value to {nice_value}")
87 out.eerror("%s\n" % str(e))
88
89 + autogroup_file = Path("/proc/self/autogroup")
90 + try:
91 + f = autogroup_file.open("r+")
92 + except EnvironmentError:
93 + # Autogroup scheduling is not enabled on this system.
94 + return
95 +
96 + with f:
97 + line = f.readline()
98 + original_autogroup_nice_value = line.split(" ")[2]
99 +
100 + # We need to restore the original nice value of the
101 + # autogroup, as otherwise the session, e.g. the
102 + # terminal where portage was executed in, would
103 + # continue running with that value.
104 + portage.atexit_register(
105 + lambda value: autogroup_file.write_text(value),
106 + original_autogroup_nice_value,
107 + )
108 +
109 + try:
110 + f.write(nice_value)
111 + except EnvironmentError as e:
112 + out = portage.output.EOutput()
113 + out.eerror(f"Failed to change autogroup's nice value to {nice_value}")
114 + out.eerror("%s\n" % str(e))
115 +
116 +
117 def ionice(settings):
118
119 ionice_cmd = settings.get("PORTAGE_IONICE_COMMAND")
120
121 diff --git a/man/make.conf.5 b/man/make.conf.5
122 index 1c72109ad..18573b5e2 100644
123 --- a/man/make.conf.5
124 +++ b/man/make.conf.5
125 @@ -1,4 +1,4 @@
126 -.TH "MAKE.CONF" "5" "May 2021" "Portage VERSION" "Portage"
127 +.TH "MAKE.CONF" "5" "Jun 2021" "Portage VERSION" "Portage"
128 .SH "NAME"
129 make.conf \- custom settings for Portage
130 .SH "SYNOPSIS"
131 @@ -1031,6 +1031,14 @@ The value of this variable will be added to the current nice level that
132 emerge is running at. In other words, this will not set the nice level,
133 it will increment it. For more information about nice levels and what
134 are acceptable ranges, see \fBnice\fR(1).
135 +.br
136 +If set and portage is run under Linux with autogroup scheduling (see
137 +\fBsched\fR(7)) enabled, then portage will set the nice value of its
138 +autogroup to PORTAGE_NICENESS. Upon exiting, portage will restore the
139 +original value. Note that if the function responsible for restoring the
140 +original value is not run, e.g., because portage's process was killed,
141 +then the autogroup will stay niced. In such a case, the value can be
142 +reset via corresponding autogroup pseudo\-file in /proc.
143 .TP
144 \fBPORTAGE_RO_DISTDIRS\fR = \fI[space delimited list of directories]\fR
145 When a given file does not exist in \fBDISTDIR\fR, search for the file