Gentoo Archives: gentoo-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-dev@l.g.o
Cc: dist-kernel@g.o, kernel@g.o, Sam James <sam@g.o>
Subject: Re: [gentoo-dev] [PATCH 6/8] kernel-build.eclass: compress xz/zstd in parallel
Date: Wed, 12 Oct 2022 04:46:12
Message-Id: 00db410812fe3614cddbce47c559b20ebdaccbd9.camel@gentoo.org
In Reply to: [gentoo-dev] [PATCH 6/8] kernel-build.eclass: compress xz/zstd in parallel by Sam James
1 On Tue, 2022-10-11 at 23:54 +0100, Sam James wrote:
2 > Note that this will only take effect if compressing
3 > modules with xz or zstd (COMPRESS_MODULE_XZ is set or
4 > COMPRESS_MODULE_ZSTD is set respectively)
5 >
6 > ... or if compiling the kernel with xz or zstd (COMPRESS_KERNEL_XZ
7 > or COMPRESS_KERNEL_ZSTD is set respectiely).
8 >
9 > Signed-off-by: Sam James <sam@g.o>
10 > ---
11 > eclass/kernel-build.eclass | 10 +++++++++-
12 > 1 file changed, 9 insertions(+), 1 deletion(-)
13 >
14 > diff --git a/eclass/kernel-build.eclass b/eclass/kernel-build.eclass
15 > index 98a385bf72c0..936258e9e8af 100644
16 > --- a/eclass/kernel-build.eclass
17 > +++ b/eclass/kernel-build.eclass
18 > @@ -29,7 +29,7 @@ esac
19 >
20 > PYTHON_COMPAT=( python3_{8..11} )
21 >
22 > -inherit python-any-r1 savedconfig toolchain-funcs kernel-install
23 > +inherit multiprocessing python-any-r1 savedconfig toolchain-funcs kernel-install
24 >
25 > BDEPEND="
26 > ${PYTHON_DEPS}
27 > @@ -90,6 +90,14 @@ kernel-build_src_configure() {
28 > ARCH=$(tc-arch-kernel)
29 > )
30 >
31 > + if type -P xz ; then
32 > + export XZ_OPT="-T$(makeopts_jobs)"
33 > + fi
34 > +
35 > + if type -P zstd ; then
36 > + export ZSTD_NBTHREADS="$(makeopts_jobs)"
37 > + fi
38 > +
39 > restore_config .config
40 > [[ -f .config ]] || die "Ebuild error: please copy default config into .config"
41 >
42
43 I'm think that we need an override for these values. Just because
44 someone is using -j12 to compile source files in parallel (which has no
45 real drawback), it doesn't mean that that person wants to use 12-thread
46 parallel xz/zstd that will actually reduce compression ratio for minimal
47 gain (I mean, most of these files won't even use parallel compression
48 because they're too small).
49
50 Not to mention that someone using -j32 distcc setup necessarily wants
51 32-threaded local compression.
52
53 --
54 Best regards,
55 Michał Górny

Replies