* [gentoo-dev] [PATCH] check-reqs.eclass: clamp MAKEOPTS for memory/RAM usage
@ 2022-01-04 22:58 99% Sam James
0 siblings, 0 replies; 1+ results
From: Sam James @ 2022-01-04 22:58 UTC (permalink / raw
To: gentoo-dev; +Cc: qa, Sam James
Crank down MAKEOPTS jobs if MAKEOPTS="-jN" is too high for the
amount of RAM available (uses amount declared as needed
in the ebuild). Typically should be ~2GB per job.
Bug: https://bugs.gentoo.org/570534
Signed-off-by: Sam James <sam@gentoo.org>
---
eclass/check-reqs.eclass | 42 +++++++++++++++++++++++++++++++++++++---
1 file changed, 39 insertions(+), 3 deletions(-)
diff --git a/eclass/check-reqs.eclass b/eclass/check-reqs.eclass
index 2130e2e349141..8c4adc8b4f121 100644
--- a/eclass/check-reqs.eclass
+++ b/eclass/check-reqs.eclass
@@ -43,6 +43,8 @@ case ${EAPI} in
*) die "${ECLASS}: EAPI=${EAPI:-0} is not supported" ;;
esac
+inherit multiprocessing
+
EXPORT_FUNCTIONS pkg_pretend pkg_setup
if [[ ! ${_CHECK_REQS_ECLASS} ]]; then
@@ -53,6 +55,13 @@ _CHECK_REQS_ECLASS=1
# @DESCRIPTION:
# How much RAM is needed? Eg.: CHECKREQS_MEMORY=15M
+# @ECLASS-VARIABLE: CHECKREQS_MEMORY_MANGLE_JOBS
+# @USER_VARIABLE
+# @DESCRIPTION:
+# Allow packages to reduce the number of multiprocessing (e.g. make, ninja) jobs
+# to lower memory usage.
+: ${CHECKREQS_MEMORY_MANGLE_JOBS=yes}
+
# @ECLASS-VARIABLE: CHECKREQS_DISK_BUILD
# @DEFAULT_UNSET
# @DESCRIPTION:
@@ -346,9 +355,36 @@ _check-reqs_memory() {
eend 0
else
eend 1
- _check-reqs_unsatisfied \
- ${size} \
- "RAM"
+
+ # Has the user allowed us to mangle their MAKEOPTS?
+ if [[ ${CHECKREQS_MEMORY_MANGLE_JOBS} == "yes" ]] ; then
+ local jobs=$(makeopts_jobs)
+
+ local estimated_max_memory=$((${actual_memory}/$(_check-reqs_get_kibibytes 1G)))
+ if [[ $((jobs*2)) -gt ${estimated_max_memory} ]] ; then
+ # Number of jobs exceeds RAM/2GB, so clamp it.
+ local new_jobs=$(($(_check-reqs_get_number ${estimated_max_memory}G)*10/20))
+
+ # This might _still_ be too big on small machines. Give up in such cases.
+ # (Users can still set the do nothing variable which is independent of this.)
+ if [[ $((new_jobs*2)) -gt ${estimated_max_memory} ]] ; then
+ _check-reqs_unsatisfied \
+ ${size} \
+ "RAM"
+ else
+ # The clamped jobs seem to be enough to satisfy the check-reqs requirement from the ebuild.
+ ewarn "Clamping MAKEOPTS jobs to -j${new_jobs} to reduce memory usage"
+ ewarn "Compiler jobs may use around ~2GB each: https://wiki.gentoo.org/wiki/MAKEOPTS"
+ ewarn "To disable this, set CHECKREQS_MEMORY_MANGLE_JOBS=no."
+
+ MAKEOPTS+=" -j${new_jobs}"
+ fi
+ fi
+ else
+ _check-reqs_unsatisfied \
+ ${size} \
+ "RAM"
+ fi
fi
else
eend 1
--
2.34.1
^ permalink raw reply related [relevance 99%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2022-01-04 22:58 99% [gentoo-dev] [PATCH] check-reqs.eclass: clamp MAKEOPTS for memory/RAM usage Sam James
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox