Re: [gentoo-dev] Value of Continuous integration vs Code Review / Pull Requests

[Alec Warner]

On Wed, May 27, 2020 at 5:16 AM Thomas Deutschmann <whissi@g.o>
wrote:

> Hi,
>
> is this really CI _vs_ Code Review? I.e. we can only have one?
>

I'll try to come back to this. We can make computers do anything, but it's
a question of limited time for me and for Gentoo as a whole.


>
> CI is nice and helpful. For example I usually don't start review until CI
> sets green light but CI alone wouldn't be enough. Thought that Gitlab will
> support same kind of hooks similar to how current CI is integrated into
> Github, not?
>

So I'm basically back to worrying about social contract here. If its a
common Gentoo workflow to:
 - Have a user fork a repo (github)
 - Submit a PR (github)
 - Gentoo CI sees the PR against ::gentoo and runs CI (github)
 - CI turns green and comments on the PR (github)
 - Human Gentoo dev reviews PR (github) [some iterations of review and
changes with PR author]
 - Human Gentoo dev downloads PR patch and applies to Gentoo (Gentoo
Hosted.)

Does this workflow meet the social contract? I would assert it does not.
Now of course Gentoo doesn't "solely rely" on this workflow and other
workflows are available; but if this is a de-facto workflow people are
using...does not that concern the community?  It concerns me! Hey we
originally were like 'nah github is OK' and now I'm coming around and
saying 'hey maybe this isn't OK in its current form.' So one idea might be
"what can we do about this particular problem."

What others have done is typically left mirrors of their projects on
github, written bots to auto-close PRs to those repos, and funneled users
somewhere else. We could do that if we wanted.


>
> Also, when I could wish something: The problem when doing review on Github
> for me is, that we usually create new revisions. Therefore we don't see
> what's changed in new revision versus previous revision. So the change to
> review looks like an entire new file was added (which is what basically
> happened). It would be cool if our solution would be aware of this and
> could handle this somehow. But I guess we would have to create our own
> solution for this...
>

So to address your first point above, there are a bunch of (sometimes
conflicting) requirements.

[Github]
A past argument for Github was "we need to be on github because this is
where the users are." I can tell you the users have not left Github and so
without us funneling them somewhere else...I mean if we are going to
continue to accept PRs on github we should probably service them (vs
auto-closing.)

[Code Review]
Basically how do we build a system where a user can go find our code, fork
it, generate a patch, send us the patch, and then be able to run CI on it
and do a review online (as opposed to on bugzilla / in-email.) Currently
this happens on github and I think there is some support for moving those
users to some other solution.

We could:
 - Build something on gitlab CE.
 - Build something on gitea + a CI solution (drone, zuul, etc.)
 - I don't think gerrit credibly meets these requirements because users
can't fork in gerrit and it requires annoying client side tooling.
 - The main problem I think with gitlab-CE is no one wants to operate it;
its too complex of a software package and there is a major fear that once
we mgirate everyone on it, something will go wrong and then we will be in
trouble. I don't have this fear with gitea because there are fewer moving
parts.
 - The main problem with gitea is that it's relatively unproven and we
already know it needs patches for ::gentoo.

[CI-for-ebuilds]
My understanding is that there are two existing CI workflows (I'm ignoring
the new thing nattka; it's what I'd consider a different workflow.)
(1) Github PRs: A bot watches for PRs against ::gentoo on Github and runs
CI on them.
(2) Post-Submit CI: Every so often we run Ci against ::gentoo and if
something looks super bad we bisect and email that the build is broken.

So right off the bat there are some gaps.
 - There is no pre-submit CI on git.gentoo.org at all, so devs can't vet
their changes there. Would anyone like this?
 - There is no flexibility; the CI you get is whatever is configured (e.g.
what mgorny set up). This isn't meant as a dig; maybe no one wants to set
up CI at all and just wants to use this stuff; I have no idea.

[Other Projects]
Portage has had CI for ages; but it runs on github and uses travis-CI
soko.git uses hosted gitlab-ci to build, run tests, and upload containers.
https://gitweb.gentoo.org/proj/catalyst.git/ has no CI, but might like some?
https://gitweb.gentoo.org/proj/gentoo-mirrorstats.git/
<https://gitweb.gentoo.org/proj/gentoo-mirrorstats.git/tree/> has no CI but
might like some?
https://gitweb.gentoo.org/proj/sandbox.git/ has no CI but might want some?
I could go on.

So to kind of summarize. If I wanted to just do [CI for other projects] we
could likely just build that now and not need to talk to most of the
community because the CI for other projects is fairly segmented. It's the
part that delivers a lot of value (to me!) but, I'd wager, little value to
the rest of the community. However the Code Review is intertwined with CI;
because I think CI brings a bunch of value into the Code Review process and
there are significant Code Review paths (bugzilla, git patches) that don't
have presubmit coverage that I think could be fixed. This is why I
separated the concerns.

-A


>
> --
> Regards,
> Thomas Deutschmann / Gentoo Linux Developer
> fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
>