| 1 |
> On 6 Nov 2022, at 08:15, Michał Górny <mgorny@g.o> wrote: |
| 2 |
> |
| 3 |
> Hi, everyone. |
| 4 |
> |
| 5 |
> Arch testing's relying on automation a lot these days. Not saying |
| 6 |
> that's bad, if it improves the state of affairs. However, I have some |
| 7 |
> concerns, based on what I've seen lately. |
| 8 |
|
| 9 |
Thanks for starting this discussion, I think others have felt this way too. |
| 10 |
|
| 11 |
> |
| 12 |
> On top of that, it seems that most of it still relies on proprietary |
| 13 |
> software and we have no clue how *exactly* it works, and it's really, |
| 14 |
> really hard to get a straight answer. |
| 15 |
> |
| 16 |
|
| 17 |
arthurzam, jsmolic, and I are using https://github.com/arthurzam/tattoo. |
| 18 |
|
| 19 |
> So, my questions are: |
| 20 |
> |
| 21 |
> 1. Is "runtime testing required" field being respected? Obviously not |
| 22 |
> every package can be (sufficiently) tested via FEATURES=test, so we've |
| 23 |
> added that fields. However, if arch testers just ignore it and push |
| 24 |
> things stable based on pure build testing... |
| 25 |
|
| 26 |
Not right now. We discussed it on #gentoo-dev maybe 2 months ago |
| 27 |
or so but concluded we needed nattka support to fix up our automation. |
| 28 |
|
| 29 |
That had two parts: |
| 30 |
1. https://github.com/projg2/nattka/issues/72 & https://github.com/projg2/nattka/pull/73 (done) |
| 31 |
2. https://github.com/arthurzam/tattoo/issues/1 (not done) |
| 32 |
|
| 33 |
> |
| 34 |
> 2. How are kernels being tested? Given the speed with which new gentoo- |
| 35 |
> sources stablereqs are handled, I really feel like "arch testing" there |
| 36 |
> means "checking if sources install", and have little to do with working |
| 37 |
> kernels. |
| 38 |
> |
| 39 |
|
| 40 |
I usually blacklist gentoo-sources because I can't actually test it, or |
| 41 |
if I do stable it, I've tried to run them for real. |
| 42 |
|
| 43 |
For gentoo-kernel, I run src_test. |
| 44 |
|
| 45 |
> 3. How does the automation handle packages that aren't trivially |
| 46 |
> installable? I recall that in the past stablereqs were stalled for |
| 47 |
> months without a single comment because automation couldn't figure out |
| 48 |
> how to proceed, and nobody bothered reporting a problem. |
| 49 |
|
| 50 |
It doesn't, really. I think at the very least we need to try do world |
| 51 |
upgrades after adding the package list to package.accept_keywords, |
| 52 |
as stuff like LLVM and GNOME bugs won't work without it (blockers etc). |
| 53 |
|
| 54 |
Best, |
| 55 |
sam |
Archives