Gentoo Archives: gentoo-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-dev@l.g.o
Cc: "Michał Górny" <mgorny@g.o>
Subject: [gentoo-dev] [PATCH v2] multiprocessing.eclass: Introduce get_nproc() to get no of CPUs
Date: Wed, 14 Dec 2016 19:15:04
Message-Id: 20161214191446.2694-1-mgorny@gentoo.org
In Reply to: [gentoo-dev] [PATCH 2/3] multiprocessing.eclass: Introduce get_nproc() to get no of CPUs by "Michał Górny"
1 Introduce get_nproc(), a portable 'nproc' wrapper. It uses either
2 'nproc' or a fallback Python multiprocessing module call to attempt to
3 determine the number of available processing units.
4
5 This can be used e.g. to determine a safe number of jobs to run when
6 MAKEOPTS specifies unlimited --jobs and the build system in question
7 does not support --load-average.
8
9 // v2: added sysctl call for BSDs
10 ---
11 eclass/multiprocessing.eclass | 32 ++++++++++++++++++++++++++++++++
12 1 file changed, 32 insertions(+)
13
14 diff --git a/eclass/multiprocessing.eclass b/eclass/multiprocessing.eclass
15 index 5a5fe9acb56a..3c5dfff2d11c 100644
16 --- a/eclass/multiprocessing.eclass
17 +++ b/eclass/multiprocessing.eclass
18 @@ -53,6 +53,38 @@ bashpid() {
19 sh -c 'echo ${PPID}'
20 }
21
22 +# @FUNCTION: get_nproc
23 +# @USAGE: [${fallback:-1}]
24 +# @DESCRIPTION:
25 +# Attempt to figure out the number of processing units available.
26 +# If the value can not be determined, prints the provided fallback
27 +# instead. If no fallback is provided, defaults to 1.
28 +get_nproc() {
29 + local nproc
30 +
31 + # GNU
32 + if type -P nproc &>/dev/null; then
33 + nproc=$(nproc)
34 + fi
35 +
36 + # BSD
37 + if [[ -z ${nproc} ]] && type -P sysctl &>/dev/null; then
38 + nproc=$(sysctl -n hw.ncpu 2>/dev/null)
39 + fi
40 +
41 + # fallback to python2.6+
42 + # note: this may fail (raise NotImplementedError)
43 + if [[ -z ${nproc} ]] && type -P python &>/dev/null; then
44 + nproc=$(python -c 'import multiprocessing; print(multiprocessing.cpu_count());' 2>/dev/null)
45 + fi
46 +
47 + if [[ -n ${nproc} ]]; then
48 + echo "${nproc}"
49 + else
50 + echo "${1:-1}"
51 + fi
52 +}
53 +
54 # @FUNCTION: makeopts_jobs
55 # @USAGE: [${MAKEOPTS}]
56 # @DESCRIPTION:
57 --
58 2.11.0