Re: [gentoo-alt] 'Continuous Integration' for Gentoo Prefix?

[Sam Pfeiffer]

Hello everyone,

I'm very excited to see you are interested in adding continuous integration!

I don't know that much about continuous integration, I've only used it
(with systems already setup for me) with in-house Jenkins servers and with
the ROS buildfarm, based on Travis CI on Github. Also a little bit of
Gitlab CI in my lab.

I did a bit of research/testing.

Given it's quite a hassle to maintain custom machines, I'd try to use some
of the free for opensource CI services. I've checked the conditions of a
few to see which could fit better:

* Gitlab CI: 2000 minutes / month
* Travis CI: Unlimited minutes / month. But only 50 minutes long per step
(like per script executed).
* Azure pipelines: Unlimited minutes / month. 360 minutes long per step
(like per script executed).

There are probably many more, but those are the ones I knew about.

Given that I wanted to give a try to Azure pipelines. And I did!

I created this repo: https://github.com/awesomebytes/gentoo_prefix_ci_test

Where I activated Azure pipelines on it. After around 15min of reading the
docs and playing around with the web-gui I got my first pipeline running.

As an initial setup I thought I would create a Docker image where I
bootstrap Gentoo Prefix from a Ubuntu 16.04 (as I'm familiar with it with
my projects).

The repo contains two important things:
1) The Dockerfile where I mainly trigger the bootstrap:
https://github.com/awesomebytes/gentoo_prefix_ci_test/blob/master/Dockerfile
2) The configuration file for Azure pipelines on what to do:
https://github.com/awesomebytes/gentoo_prefix_ci_test/blob/master/azure-pipelines.yml

I've implemented here that it tries to build Gentoo Prefix, and whatever
the result, it uploads a Docker image to my DockerHub account with the
results. This implies that:
If the bootstrap is successful, one can just [docker pull] and [docker run]
the image to play with Gentoo Prefix.
If the bootstrap is unsuccessful, one can just [docker pull] and [docker
run] to find oneself in the exact state of the system after the bootstrap
command. And one can recover the full console log from the Azure pipelines
web interface (even tho it would be nice to find out how to post it
publicly straight away).

If all goes well in a few hours anyone will be able to find in my DockerHub
account said image (most probably the failed one), just doing:
docker pull awesomebytes/gentoo_prefix_latest_image:latest
docker run -it gentoo_prefix_latest_image /bin/bash
You'll be inside of a Ubuntu 16.04 box with a user called 'user' and with
all the bootstrapped stuff in /tmp/gentoo.

As curiosity, I checked the machines I got served as 'agents' to run my
jobs, and they were of the kind:

CPUs: 2x Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
RAM: 7GB
Disk: 94GB free disk space

More than enough to bootstrap Gentoo Prefix!

I don't know if this is the way to go. But at least is interesting to have
it in mind.


On Tue, Nov 27, 2018 at 12:01 AM Benda Xu <heroxbd@g.o> wrote:

> Hi Sam,
>
> Sam Pfeiffer <sammypfeiffer@×××××.com> writes:
>
> > With Azure announcing unlimited minutes on CI/CD for open source
> > projects:
> >
> https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> >
> > Even bootstrapping Gentoo prefix, with pieces of software like gcc
> > taking very long to compile, is possible.
> >
> > The point is: I have been trying to build Gentoo Prefix during the
> > last days after a few months of break since the last time I touched
> > the system. And it's failing. I haven't managed yet to bootstrap it
> > completely. I feel there is no CI/CD setup to catch these issues and
> > be able to offer a working version of Gentoo Prefix at any time.
>
> I completely agree with you.  I hope you can carry on this project to
> setup proper CI for Gentoo Prefix.  I am all in for help, portage/ebuild
> mentoring and coorperation.
>
> A CI for Gentoo Prefix has been on my list for ages.  Thank you for
> triggering this.
>
> Yours,
> Benda
>
>

-- 

*Sammy Pfeiffer*
PhD Candidate at The Magic Lab within UTS.