Gentoo Archives: gentoo-dev

From: Agostino Sarubbo <ago@g.o>
To: gentoo-dev@l.g.o
Cc: "Michał Górny" <mgorny@g.o>
Subject: Re: [gentoo-dev] Disturbing state of arch testing in Gentoo
Date: Sun, 06 Nov 2022 18:51:09
Message-Id: 3218817.44csPzL39Z@spectre
In Reply to: [gentoo-dev] Disturbing state of arch testing in Gentoo by "Michał Górny"
1 On domenica 6 novembre 2022 09:15:40 CET Michał Górny wrote:
2 > On top of that, it seems that most of it still relies on proprietary
3 > software and we have no clue how *exactly* it works, and it's really,
4 > really hard to get a straight answer.
5
6 I'm speaking for myself. I still use getatoms.py to fetch 'doable' stablereqs (it is on my todo
7 to switch to nattka). And I have a script the *simply* does emerge over the list of the
8 packages.
9 There is nothing obscure in it.
10
11
12 > So, my questions are:
13 >
14 > 1. Is "runtime testing required" field being respected? Obviously not
15 > every package can be (sufficiently) tested via FEATURES=test, so we've
16 > added that fields. However, if arch testers just ignore it and push
17 > things stable based on pure build testing...
18
19 sam already provided the right answer. In addition, when we introduced runtime_testing
20 and package_list fields we requested support in pybugz:
21
22 https://github.com/williamh/pybugz/issues/105[1]
23
24 There is no trace (into the github ticket) about runtime testing field because I discussed/
25 requested over irc.
26
27 > 2. How are kernels being tested? Given the speed with which new gentoo-
28 > sources stablereqs are handled, I really feel like "arch testing" there
29 > means "checking if sources install", and have little to do with working
30 > kernels.
31
32 For amd64, I boot into the new kernel to verify that at least it boots. For other 'exotic'
33 arches, since there is a lack of hardware, the rule was to verify that at least it builds (install
34 if we want to use the right word).
35 If you think that build only is not appropriate, I can skip kernel stablereqs.
36
37
38 > 3. How does the automation handle packages that aren't trivially
39 > installable? I recall that in the past stablereqs were stalled for
40 > months without a single comment because automation couldn't figure out
41 > how to proceed, and nobody bothered reporting a problem.
42
43 I skip them from automation and from time to time I handle it manually.
44
45
46 Agostino
47
48
49
50 --------
51 [1] https://github.com/williamh/pybugz/issues/105