Re: [gentoo-dev] Value of Continuous integration vs Code Review / Pull Requests

[Alec Warner]

On Wed, May 27, 2020 at 1:09 AM Brian Dolbec <dolsen@g.o> wrote:

> On Tue, 26 May 2020 20:24:56 -0700
> Alec Warner <antarus@g.o> wrote:
>
> > The TL;DR is that a crack team of infra-folks[0] have been putting
> > together demos of CI services and things like gitlab / gitea / gerrit
> > and so on.
> >
> > Some of these come in combined (e.g. gitlab offers repo hosting, code
> > review / pull reqs, CI services, and deploy services.) Some of these
> > are piecemeal (e.g. gerrit has code review, zuul has CI) and gitea
> > offers repo-hosting but CI is separate (e.g. drone.)
> >
> > On the infra-side, I think we are pretty happy with repo-hosting
> > (gitolite) and repo-serving (gitweb). We are missing a CI piece and a
> > pull-request piece. Most of the users using PRs use either a gitlab
> > or github mirror.
> >
> > I think the value of CI is pretty obvious to me (and I see tons of use
> > cases in Infra.) We could easily build CI into our current repository
> > solution (e.g. gitolite.) However gitolite doesn't really support PRs
> > in a uniform way and so CI is mostly for submitted code; similar to
> > the existing ::gentoo repo CI offered by mgorny.
> >
> > If we build a code review solution (like gitea / gerrit) would people
> > use it? Would you use it if you couldn't merge (because the code
> > review solution can't gpg sign your commits or merges) so a tool like
> > the existing pram tool would be needed to merge?
> >
> > -A
> >
> > [0] Mostly arzano, if I'm honest. I am just the point-y haired
> > manager in this effort.
>
> There are several CI systems that could be used.  As for CI, my
> experience has been with buildbot.  It would be fairly straightforward
> to integrate with the current gitolite/gitweb hosting.
> It is also extremely flexible in its capabilities, although can be
> difficult to master.  It has a good web interface, but it can even be
> run via it's API's using curses, python, bash,....   There is a
> buildbot-travis plugin which is capable of running existing .travis file
> configurations.  It also extends its capabilities to include custom
> buildbot step definitions.  I have not packaged it yet for gentoo, but
> it is on my todo list. One of buildbot's capabilities is the ability to
> run tests/builds on multiple workers (different arches or whatever)
> both in parallel or series.  It could be made to integrate with our
> bugzilla using the python client (pybugs was it?).
>

My understanding is that CI systems are all quite similar. We have
configured gitlab-ci, drone, and zuul as tests and I am familiar with
travis-ci and buildbot from other projects. I'm less concerned about this
aspect tbh. I'm also not super concerned about packaging. E.g. for
gitlab-runner (the worker portion of gitlab-ci) we just deploy upstream
containers and don't package them in ::gentoo at all. Our onprem gitlab is
a container solution; gitea is a container solution; our SSO IDP is a
container solution; gerrit is currently a container solution. These don't
bother me (too much, ugh zuul uses zookeeper? ugh.)

The major differentiators for CI appear to be:
 - SCM support (we currently only really care about git compatible systems,
but some contenders only support some providers.)
 - builders / workers (how do they deploy). For example gitlab has a
container based work executor while zuul uses ansible; I suspect
 - Authentication (ideally should support SAML or openID so we can
integrate with our alpha sso solution for Gentoo.)
 - The webUI; e.g is the solution horrible to use / interact with. Hard to
say without a trial, IMHO.

-A


>
> But that still leaves a PR/code review option.
>
>