Gentoo Archives: gentoo-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-dev-announce <gentoo-dev-announce@l.g.o>
Cc: gentoo-dev <gentoo-dev@l.g.o>
Subject: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB
Date: Sun, 17 Dec 2017 13:21:24
1 Hello, everyone.
3 It's my pleasure to announce that with a majority vote the QA team has
4 accepted a new policy. The accepted wording is:
6 Total size of 'files' subdirectory of a package should not be larger
7 than 32 KiB. If the package needs more auxiliary files, they should
8 be put into SRC_URI e.g. via tarballs.
10 (the total size being computed as a sum of apparent file sizes)
12 The relevant policy vote is finishing at bug #633758 [1]. The CI reports
13 [2] were updated to report packages whose 'files' directories exceed
14 64 KiB, to avoid adding many new warnings at once. The limit will
15 be lowered down to 32 KiB as packages are fixed to comply with the new
16 policy.
18 At the same time, I would like to explicitly remind developers that
19 the spirit of the policy is 'do not let "files" grow large', not 'make
20 sure you're one byte less than 32769.' Do not argue that your package
21 exceeds the limit only by few bytes -- even if it gets close to the
22 limit, then it means it's way too large.
25 Motivation
26 ==========
28 Repoman & pkgcheck so far checked for a single file in 'files' directory
29 that exceeded 20 KiB in size. This check has been criticized multiple
30 times, most notably because it was triggered on packages with a single
31 large-ish file but at the same time permitted packages to have a lot of
32 small-ish files, with the latter consuming much more space. Furthermore,
33 some developers explicitly worked around the check by splitting large
34 patches into two smaller parts.
36 The new policy aims to serve the same purpose but be more fair
37 by counting the total size of all files.
40 Rationale
41 =========
43 At this moment, syncing the repository implies fetching 'files'
44 directories of all packages, even though the relevant files are used
45 only when a ebuild referencing them is being built. This means that our
46 users fetch many files that they will never use -- either because they
47 don't need the package in question, or because the file belongs
48 to an old version.
50 For example, 'du -h app-shells/bash/files' states 232K while only three
51 of those files are used by the newest version, and everything else are
52 patches for old versions. And in case of bash, we're keeping those
53 versions pretty much 'forever'.
55 The new policy mostly targets large patchsets and files relevant to old
56 package versions. By removing them from the repository, we're hoping to
57 reduce the growth of its size a bit and reduce the amount of data
58 transferred via rsync.
61 [1]:
62 [2]:
64 --
65 Best regards,
66 Michał Górny