Re: [gentoo-dev] Disturbing state of arch testing in Gentoo

[John Helmert III]

On Sun, Nov 06, 2022 at 09:15:40AM +0100, Michał Górny wrote:
> Hi, everyone.
>
> Arch testing's relying on automation a lot these days.  Not saying
> that's bad, if it improves the state of affairs.  However, I have some
> concerns, based on what I've seen lately.
>
> On top of that, it seems that most of it still relies on proprietary
> software and we have no clue how *exactly* it works, and it's really,
> really hard to get a straight answer.

As far as I can tell, there's ONE person relying completely on a
proprietary arch testing system.

Ago, could you comment on this? What's blocking you from open sourcing
your software?

I'll also point out that you removed the Github repository that you
used to tell people to report issues with your CI at, while there were
several outstanding issues. Why? I have two (of three) issue titles in
my browser history, but as I recall you never touched them:

2. Release source code
3. More information on the bug

[1] https://github.com/asarubbo/ci

> So, my questions are:
>
> 1. Is "runtime testing required" field being respected?  Obviously not
> every package can be (sufficiently) tested via FEATURES=test, so we've
> added that fields.  However, if arch testers just ignore it and push
> things stable based on pure build testing...
>
> 2. How are kernels being tested?  Given the speed with which new gentoo-
> sources stablereqs are handled, I really feel like "arch testing" there
> means "checking if sources install", and have little to do with working
> kernels.
>
> 3. How does the automation handle packages that aren't trivially
> installable?  I recall that in the past stablereqs were stalled for
> months without a single comment because automation couldn't figure out
> how to proceed, and nobody bothered reporting a problem.
>
> --
> Best regards,
> Michał Górny
>
>