* [gentoo-dev] some multilib-minimal enhancements [5/6]: add frob to control phase parallelization
@ 2013-12-11 21:19 99% Greg Turner
0 siblings, 0 replies; 1+ results
From: Greg Turner @ 2013-12-11 21:19 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1 bytes --]
[-- Attachment #2: 005-MULTILIB_PARALLEL_PHASES.patch --]
[-- Type: text/x-patch, Size: 4866 bytes --]
This patch adds a new frob, MULTILIB_PARALLEL_PHASES, to
multlib-minimal.eclass, which implements eclass-consumer-selectable
parallelization of src_configure, src_compile, and src_test.
By default, all parallelization is deactivated, which represents
a change from the previous gentoo-x86 behavior (which was
parallelizing src_configure automatically).
--- 004-multilib-phase-all/multilib-minimal.eclass 2013-12-03 02:54:40.045335905 -0800
+++ 005-MULTILIB_PARALLEL_PHASES/multilib-minimal.eclass 2013-12-03 02:59:33.687448429 -0800
@@ -45,8 +45,8 @@
# for more information.
#
# Another snag you may encounter is that, in many circumstances, it is possible
-# for variables to flow between multilib_<phase> functions. This is because
-# no subshell is utilized.
+# for variables to flow between multilib_<phase> functions. This is because,
+# unless you parallelize the phase in question, no subshell is utilized.
# Extreme care must sometimes be taken to ensure that different ABI's don't
# trample each other's variables.
#
@@ -122,6 +122,58 @@ case ${EAPI:-0} in
*) die "EAPI=${EAPI} is not supported" ;;
esac
+# @ECLASS-VARIABLE: MULTILIB_PARALLEL_PHASES
+# @DEFAULT-UNSET
+# @DESCRIPTION:
+# multilib-minimal.eclass consumers may set this to a string containing
+# a space-separated list of phase-function names, like so:
+#
+# @CODE@
+# MULTILIB_PARALLEL_PHASES="src_configure src_test"
+# @CODE@
+#
+# For each (supported) phase-function name in the list, the corresponding
+# multilib-minimal_<phase> phase function will execute in parallel, in
+# the sense that, within limits (set by multiprocessing.eclass) the
+# multilib_<phase> function invocations (or, if none is present, the
+# built-in default implementation) for each ABI will run simultaneously.
+#
+# Any phase-function-names not specified are executed serially, with the
+# native ABI coming last.
+#
+# It is OK to add extra "bonus" whitespace or duplicated values.
+# Changes take effect immediately, so it may be set before or after
+# inheriting multilib-minimal, or even during execution of a phase function
+# (but not in parallelized multilib_<phase> functions; see below).
+#
+# By default, MULTILIB_PARALLEL_PHASES is empty. Consuming eclasses could
+# override this default by setting a new default before the inherits clause.
+# For example,
+#
+# @CODE
+# : {MULTILIB_PARALLEL_PHASES:="src_configure"}
+# inherit multilib-minimal
+# @CODE
+#
+# would create a default behavior of parallel src_configure. The question
+# of whether, in practice, eclasses ever should do so is potentially
+# complicated; you'll have to figure that out for yourself.
+#
+# Supported phase-function-names are: src_configure, src_compile, and src_test.
+# All other values are silently ignored.
+#
+# Note that parallel execution can lead to ugly -- and sometimes very, very ugly --
+# console output at build-time. Consumers of multilib-minimal.eclass
+# may wish to consider activating less verbose build-time output modes, if they are
+# readily available, to mitigate this effect, although doing so goes against a
+# long-standing Gentoo tradition and might make debugging a hassle for bugzilla
+# and IRC participants, so use your best judgement.
+#
+# A handy per-ABI log is kept in ${T}/build-${ABI}.log by multibuild.eclass.
+# Debugging and bug-reporting may also benefit (unless your bug specifically
+# pertains to parallelization) from setting MAKEOPTS=-j1, which will prevent
+# parallelization entirely.
+
# @ECLASS-VARIABLE: MULTILIB_INSECURE_INSTALL
# @DEFAULT-UNSET
# @DESCRIPTION:
@@ -161,7 +213,12 @@ multilib-minimal_src_configure() {
if declare -f multilib_src_configure_all > /dev/null ; then
multilib_src_configure_all
fi
- multilib_foreach_abi multilib-minimal_abi_src_configure
+
+ if has src_configure ${MULTILIB_PARALLEL_PHASES} ; then
+ multilib_parallel_foreach_abi multilib-minimal_abi_src_configure
+ else
+ multilib_foreach_abi multilib-minimal_abi_src_configure
+ fi
}
multilib-minimal_src_compile() {
@@ -179,7 +236,12 @@ multilib-minimal_src_compile() {
popd >/dev/null || die
}
- multilib_foreach_abi multilib-minimal_abi_src_compile
+ if has src_compile ${MULTILIB_PARALLEL_PHASES} ; then
+ multilib_parallel_foreach_abi multilib-minimal_abi_src_compile
+ else
+ multilib_foreach_abi multilib-minimal_abi_src_compile
+ fi
+
if declare -f multilib_src_compile_all > /dev/null ; then
multilib_src_compile_all
fi
@@ -200,7 +262,12 @@ multilib-minimal_src_test() {
popd >/dev/null || die
}
- multilib_foreach_abi multilib-minimal_abi_src_test
+ if has src_test ${MULTILIB_PARALLEL_PHASES} ; then
+ multilib_parallel_foreach_abi multilib-minimal_abi_src_test
+ else
+ multilib_foreach_abi multilib-minimal_abi_src_test
+ fi
+
if declare -f multilib_src_test_all >/dev/null ; then
multilib_src_test_all
fi
^ permalink raw reply [relevance 99%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2013-12-11 21:19 99% [gentoo-dev] some multilib-minimal enhancements [5/6]: add frob to control phase parallelization Greg Turner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox