Re: [gentoo-alt] Re: 'Continuous Integration' for Gentoo Prefix?

[Fabian Groffen]

Hey Sam,

Examining some of your build logs, I wonder if you are aware you can do
this:

  ./bootstrap-prefix.sh ${EPREFIX} noninteractive

instead of forcing input to the script.

Perl currently fails it seems in the logs, though the bootstrap is
reported to be successful.  Is this by design?

Thanks,
Fabian

On 03-12-2018 11:53:02 +0100, Fabian Groffen wrote:
> Cool!  Thanks a lot!
>
> Fabian
>
> On 03-12-2018 21:48:15 +1100, Sam Pfeiffer wrote:
> > Hello,
> >
> > Just wanted to update you up to where I got.
> >
> > Now I have two working repositories:
> >
> > * [1]https://github.com/awesomebytes/gentoo_prefix_ci
> >
> > * [2]https://github.com/awesomebytes/gentoo_prefix_ci_32b
> >
> > They have continuous integration setup with Azure pipelines where every night
> > they will try to bootstrap Gentoo Prefix on amd64 and x86 using Docker images.
> >
> > As the READMEs state, this is currently done in 3 steps. This is done for two
> > reasons. First, the 6h limit of one job running. Secondly, to be able to have
> > intermediate state Docker images to maybe try to fix the current issues
> > bootstrapping Gentoo Prefix in a more elegant way.
> >
> > Using as an example the amd64 build:
> >
> > On the releases
> > section: [3]https://github.com/awesomebytes/gentoo_prefix_ci_32b/releases one
> > can find .tar.gz files with the latest (currently done by hand, I'll automate
> > that soon on successful builds) Gentoo Prefix that bootstrapped. It's
> > bootstrapped in /tmp/gentoo and explained how to use it as I explained in this
> > email thread before.
> >
> > On the builds
> > page: [4]https://dev.azure.com/12719821/12719821/_build?definitionId=2 one can
> > find the full logs of every step.
> >
> > This fulfils my immediate needs, now I'll need to spend some time doing
> > something similar to emerge all the stuff I need for [5]ros_overlay and offer a
> > binary repo. But I'm open to talk about what I did, improve it, maybe move it
> > somewhere else... You let me know!
> >
> > P.S.: Most of the work, if not all, is documented in bug [6]#668940 and more
> > detailed and in order in [7]this notepad originally from Olivier Huber.
> >
> > P.S.2: The help I got from the people in the IRC at #gentoo-prefix was great.
> >
> > On Tue, Nov 27, 2018 at 8:21 PM Michael Haubenwallner <[8]haubi@g.o>
> > wrote:
> >
> > > On 11/27/2018 09:37 AM, Sam Pfeiffer wrote:
> > > > On Tue, Nov 27, 2018 at 7:20 PM Fabian Groffen <[9]grobian@g.o
> > > <mailto:[10]grobian@g.o>> wrote:
> > > >
> > > > > I don't want to depress this entire discussion, but it would be really
> > > > > nice if we could somehow interact with special machines people have at
> > > > > their company or at home.  Prefix needs testing on many different
> > > > > machines (non-Linux) which usually don't exist in docker images.
> >
> > > I second this - and let me add a further aspect here:
> > > What I know from buildbot setup is that the master does provide (mostly shell)
> > > commands to be executed on the slave. This is fine as long as there is limited
> > > visibility for the master. But when a public buildbot master is being
> > > hijacked,
> > > it feels too easy to execute malicious commands even on the slave machines.
> >
> > > So over a buildbot like setup, I would prefer a Jenkins like setup, where the
> > > master does provide only trigger information to slaves. And even more
> > > appealing
> > > would be a standalone slave setup, where the master does just receive the
> > > build
> > > logs for the public, without access to slave machines at all.
> >
> > > > That's alright, we can use QEMU for some more esoteric hardware platforms,>
> > > if it's an OS that runs on a normal amd64/x86 computer a Docker image can be
> > > > built (I'm not an expert but there are images to learn how to do it).> Or in
> > > the worst case we can create an old-school VM for those weird OS
> > > > and automate the interaction with it (I did it for a robot by dumping all
> > > > disk once and creating a VM from it, it worked ok).
> >
> > > Well... there's a bunch of OSs I fail to imagine the use of cloud driven
> > > hardware for them, like hppa-hpux or ia64-hpux for past ones, and ppc-aix,
> > > ppc-macos, sparc-solaris, arm-linux or m68k-mint for current ones.
> >
> > > > > That said, focussing on the (usually fast) boxes like this to catch
> > > > > dependency problems and more is useful.  In the case below it looks like
> > > > > the ld-wrapper is having issues.  Would it be possible to enter the
> > > > > environment for that failed run?
> > > >
> > > > Glad you see the use of it :) Yeah as I mentioned in the previous mail,
> > > > having docker installed in your machine, to access that exact environment
> > > > after the failed bootstrap just do:
> > > >
> > > > # This will download the image to your machine (it may take a bit of time if
> > > its the first time you use docker its around 1GB of data I think)
> > > > docker pull awesomebytes/gentoo_prefix_latest_image
> > > > # This will drop you into a shell in that environment, ready to play!
> > > > docker run -it awesomebytes/gentoo_prefix_latest_image /bin/bash
> > > >
> > > > When you are done you can just type exit.
> >
> > > Nevertheless, having the breaking environment as a docker image where
> > > possible (true for the major OSs we support) really is awesome!
> >
> > > /haubi/
> >
> > --
> >
> > Sammy Pfeiffer
> > PhD Candidate at The Magic Lab within UTS.
> >
> >
> >
> >  References:
> >    1. https://github.com/awesomebytes/gentoo_prefix_ci
> >    2. https://github.com/awesomebytes/gentoo_prefix_ci_32b
> >    3. https://github.com/awesomebytes/gentoo_prefix_ci_32b/releases
> >    4. https://dev.azure.com/12719821/12719821/_build?definitionId=2
> >    5. https://github.com/ros/ros-overlay
> >    6. https://bugs.gentoo.org/668940
> >    7. https://pad.crans.org/p/gentoo-prefix
> >    8. mailto:haubi@g.o
> >    9. mailto:grobian@g.o
> >   10. mailto:grobian@g.o
> >
> > read_char: errno==EILSEQ; invalid byte sequence for UTF-8:
> --
> Fabian Groffen
> Gentoo on a different level



-- 
Fabian Groffen
Gentoo on a different level