Gentoo Archives: gentoo-dev

From: Matthias Maier <tamiko@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB
Date: Mon, 18 Dec 2017 15:12:24
Message-Id: 87tvwocbp0.fsf@gentoo.org
In Reply to: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB by "Michał Górny"
1 On Sun, Dec 17, 2017, at 07:21 CST, Michał Górny <mgorny@g.o> wrote:
2
3 > Hello, everyone.
4 >
5 > It's my pleasure to announce that with a majority vote the QA team has
6 > accepted a new policy. The accepted wording is:
7 >
8 > Total size of 'files' subdirectory of a package should not be larger
9 > than 32 KiB. If the package needs more auxiliary files, they should
10 > be put into SRC_URI e.g. via tarballs.
11
12 > Rationale
13 > =========
14 >
15 > At this moment, syncing the repository implies fetching 'files'
16 > directories of all packages, even though the relevant files are used
17 > only when a ebuild referencing them is being built. This means that our
18 > users fetch many files that they will never use -- either because they
19 > don't need the package in question, or because the file belongs
20 > to an old version.
21 >
22 > For example, 'du -h app-shells/bash/files' states 232K while only three
23 > of those files are used by the newest version, and everything else are
24 > patches for old versions. And in case of bash, we're keeping those
25 > versions pretty much 'forever'.
26 >
27 > The new policy mostly targets large patchsets and files relevant to old
28 > package versions. By removing them from the repository, we're hoping to
29 > reduce the growth of its size a bit and reduce the amount of data
30 > transferred via rsync.
31
32
33 Assuming that only the true byte size of files are relevant and summing
34 up via
35
36 du -sb */*/files | awk '{sum+=$1} END {print sum}'
37
38 Currently gives roughly 29MB of data in the files directory. (For
39 comparison, the complete portage tree is roughly 219MB.)
40
41 Removing all "32kb offenders" results in 26MB for all files
42 directories. More aggressively, removing all "20kb offenders" results in
43 22MB.
44
45 I hardly doubt that these (at most) 10MB of saving we're talking about
46 are really relevant for rsync or git (with clone-depths 1). I haven't
47 estimated the growth of the repository, though.
48
49 Best,
50 Matthias