Re: [gentoo-dev] Request for help: distributed pull request CI (pkgcheck)

[Mathy Vanvoorden]

Hi,

I have some experience with CI and am willing to help out, I'll talk to you
on IRC.

Mathy

On Dec 7, 2016 19:57, "Michał Górny" <mgorny@g.o> wrote:

> Hello, everyone.
>
> TL;DR: I'm looking for some help to get pull-request CI running
> distributed, i.e. on multiple hosts.
>
>
> Quick history
> -------------
>
> The repo-mirror-ci project pretty much started as a simple cronjob
> running on hardware contributed to the purpose by Todd Goodman
> (considering how much this hardware has done for Gentoo, I'd suggest
> giving him a honorary citizenship of Gentoo or something like
> that ;-)). I can't say they were terrible but they weren't perfect
> either. Due to lack of time, I mostly focused on improving them
> moderately and adding new features.
>
> Today, everything is still running on the same hardware. While
> the hardware is real good, and I'm doing some effort to improve
> the performance of the tools, I think it's quite close to its limits.
>
> Right now, it is running three tasks every 20 minutes: updating repo
> mirrors, running pkgcheck on Gentoo and processing one pull request.
> However, if those tasks delay beyond those 20 minutes, the poor man's
> scheduling based on cron + lockfiles may cause some of the executions
> to be skipped.
>
> In other words, the best case is processing 3 pull requests / h. This
> is starting to be no longer sufficient, and causing significant delays
> during periods of high contributor activity -- while my goal would be to
> provide the results ASAP -- 10-20 minutes would be perfect.
>
>
> The goal and needs
> ------------------
>
> I think the best way forward would be to start distributing tasks
> between more systems. While I don't really have time to make whole
> repo-mirror-ci distributed, I think it should be feasible to start
> splitting pull request processing out of it.
>
> I would use some help to achieve that, esp. wrt distributed task
> scheduling.
>
> Can you think of any tools that could help me get the task done easily
> and with as little of reinventing the wheel as possible? Right now, I
> have just a lot of trivial shell script that checks pull request for
> changes, checks them out, runs 'pmaint regen', pkgcheck, publishes all
> kinds of results and statuses, and also compares QA results to
> determine new failures created by the PR [1].
>
> As I see it, checking for changes and submitting the results should
> stay on the current host. However, it should be able to run all
> the actual work on slaves and retrieve the results from them. I would
> appreciate all the help with implementing that, plus possibly some
> degree of control to make it possible to update pkgcheck on slaves
> easily.
>
> Please let me know if you're interested in helping. Thanks.
>
> [1]:https://bitbucket.org/mgorny/repo-mirror-ci/src/
> master/pull-requests.job
>
> --
> Best regards,
> Michał Górny
>