Re: [gentoo-dev] [PATCH 2/3] multiprocessing.eclass: Introduce get_nproc() to get no of CPUs

[Andrew Savchenko]

On Tue, 13 Dec 2016 10:36:15 +0100 Michał Górny wrote:
> Introduce get_nproc(), a portable 'nproc' wrapper. It uses either
> 'nproc' or a fallback Python multiprocessing module call to attempt to
> determine the number of available processing units.
>
> This can be used e.g. to determine a safe number of jobs to run when
> MAKEOPTS specifies unlimited --jobs and the build system in question
> does not support --load-average.
> ---
>  eclass/multiprocessing.eclass | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
>
> diff --git a/eclass/multiprocessing.eclass b/eclass/multiprocessing.eclass
> index 5a5fe9acb56a..0d241cdc15b6 100644
> --- a/eclass/multiprocessing.eclass
> +++ b/eclass/multiprocessing.eclass
> @@ -53,6 +53,31 @@ bashpid() {
>  	sh -c 'echo ${PPID}'
>  }
>
> +# @FUNCTION: get_nproc
> +# @USAGE: [${fallback:-1}]
> +# @DESCRIPTION:
> +# Attempt to figure out the number of processing units available.
> +# If the value can not be determined, prints the provided fallback
> +# instead. If no fallback is provided, defaults to 1.
> +get_nproc() {
> +	local nproc
> +
> +	if type -P nproc &>/dev/null; then
> +		# GNU
> +		nproc=$(nproc)
> +	elif type -P python &>/dev/null; then
> +		# fallback to python2.6+
> +		# note: this may fail (raise NotImplementedError)
> +		nproc=$(python -c 'import multiprocessing; print(multiprocessing.cpu_count());' 2>/dev/null)

This is not portable. E.g. paludis users can have python-less
system. Adding dev-lang/python to DEPEND will be also a bad idea,
since this is quite heavy dependency.

Since on Linux boxes nproc is from coreutils, which is in @system,
so looks like only *bsd setups are the problem. On FreeBSD
  sysctl -a | egrep -i 'hw.ncpu'
should be sufficient solution.

> +	fi
> +
> +	if [[ -n ${nproc} ]]; then
> +		echo "${nproc}"
> +	else
> +		echo "${1:-1}"
> +	fi
> +}
> +
>  # @FUNCTION: makeopts_jobs
>  # @USAGE: [${MAKEOPTS}]
>  # @DESCRIPTION:


Best regards,
Andrew Savchenko