[gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB

[Michał Górny]

Hello, everyone.

It's my pleasure to announce that with a majority vote the QA team has
accepted a new policy. The accepted wording is:

  Total size of 'files' subdirectory of a package should not be larger
  than 32 KiB. If the package needs more auxiliary files, they should
  be put into SRC_URI e.g. via tarballs.

(the total size being computed as a sum of apparent file sizes)

The relevant policy vote is finishing at bug #633758 [1]. The CI reports
 [2] were updated to report packages whose 'files' directories exceed
64 KiB, to avoid adding many new warnings at once. The limit will
be lowered down to 32 KiB as packages are fixed to comply with the new
policy.

At the same time, I would like to explicitly remind developers that
the spirit of the policy is 'do not let "files" grow large', not 'make
sure you're one byte less than 32769.' Do not argue that your package
exceeds the limit only by few bytes -- even if it gets close to the
limit, then it means it's way too large.


Motivation
==========

Repoman & pkgcheck so far checked for a single file in 'files' directory
that exceeded 20 KiB in size. This check has been criticized multiple
times, most notably because it was triggered on packages with a single
large-ish file but at the same time permitted packages to have a lot of
small-ish files, with the latter consuming much more space. Furthermore,
some developers explicitly worked around the check by splitting large
patches into two smaller parts.

The new policy aims to serve the same purpose but be more fair
by counting the total size of all files.


Rationale
=========

At this moment, syncing the repository implies fetching 'files'
directories of all packages, even though the relevant files are used
only when a ebuild referencing them is being built. This means that our
users fetch many files that they will never use -- either because they
don't need the package in question, or because the file belongs
to an old version.

For example, 'du -h app-shells/bash/files' states 232K while only three
of those files are used by the newest version, and everything else are
patches for old versions. And in case of bash, we're keeping those
versions pretty much 'forever'.

The new policy mostly targets large patchsets and files relevant to old
package versions. By removing them from the repository, we're hoping to
reduce the growth of its size a bit and reduce the amount of data
transferred via rsync.


[1]:https://bugs.gentoo.org/633758
[2]:https://qa-reports.gentoo.org/output/gentoo-ci/output.html

-- 
Best regards,
Michał Górny