[gentoo-dev] verifying commits via server-side git pre-receive hook

[Tim Harder <radhermit@gentoo.org>]

Hi all,

I've been working for a while on pkgcraft[1] and related tooling that
could have the potential to run various CI checks server-side as a
pre-receive git hook verifying commits. Previously, this wasn't at all
possible due to portage/repoman and pkgcore/pkgcheck being too slow to
feasibly run and give feedback within a second or two of interactively
pushing.

For more information on benchmarks, see the related sections in the
discussions of pkgcraft's metadata generation[2] and pkgcruft[3]
comparing them to similar functionality from portage and pkgcore. Also,
note that a majority of commits only involve scanning a limited number
of packages which is much faster than scanning the entire tree.

However, even with all that making it semi-feasible for most commits
(except perhaps those doing tree-wide changes or altering eclasses with
extensive usage) much work has to be done on the infra and git-hook
service side. First off, the server-side git repo is probably hosted in
a bare format so the git hook would need to notify a scanning service
that runs remotely on a relatively powerful machine.

In terms of service design, my high-level thoughts are as follows: the
scanning service tracks repo changes using some copy-on-write solution
that supports snapshots such as lvm2 or btrfs on top of a fast, backing
I/O solution (direct NVME or tmpfs). This would be leveraged to keep
track of the most recent tree state with metadata layered in. As commits
come in they would be put on a queue, from which the most recent would
be pulled, checked out on top of the most recent snapshot, tested, and
merged back into the most recent snapshot when scanning passes.

Alternatively, Gentoo could officially move to a merge queue approach
which would alleviate a lot of this complexity, but also bring up
questions of how merge commits and signing would work among other
issues.

 From all this, my main questions to the dev community are the following:

- Do you see value in running a service that can reject commits due to
   issues like invalid metadata during `git push`? 

- Would the project be open to moving to a merge queue model?

- Are there others who would be interested in helping with the
   development, testing, and maintenance of a git-hook service if that is
   the chosen path?

Thanks,
Tim

[1]: https://github.com/pkgcraft
[2]: https://pkgcraft.github.io/posts/metadata-cache-generation/
[3]: https://pkgcraft.github.io/posts/evolving-qa-tooling/