1 |
On 17/11/16 20:16, Rich Freeman wrote: |
2 |
> On Thu, Nov 17, 2016 at 2:16 AM, Michael Palimaka <kensington@g.o> wrote: |
3 |
>> |
4 |
>> In cases where all USE flags combinations are not being tested, it is |
5 |
>> still recommended to test: |
6 |
>> * with all USE flags enabled |
7 |
>> * with all USE flags disabled |
8 |
>> * the default USE flag settings |
9 |
>> |
10 |
> |
11 |
> I imagine that in practice only the last of these really tends to get tested. |
12 |
> |
13 |
>> * A leaf package such as {{package|kde-apps/kcalc}} may not require any |
14 |
>> runtime testing at all |
15 |
> |
16 |
> I'm not really a big fan of this, but if we REALLY didn't want to do |
17 |
> any runtime testing on a package then we should have some way to tag |
18 |
> the package as such, and then have some way to do automated |
19 |
> build-test-only stabilization. If you aren't doing runtime testing |
20 |
> then manual stabilization adds zero value. |
21 |
|
22 |
How much value do you think we gain from runtime testing a package like |
23 |
kcalc as part of the stabilisation process, considering that it already |
24 |
sat in ~arch for at least 30 days? |
25 |
|
26 |
Also, based on conversations with various arch team members, my |
27 |
understanding is that a lot of stabilisations that happen right now are |
28 |
already build-only. |
29 |
|
30 |
> |
31 |
> Overall though the writeup was good and maybe it will trigger some |
32 |
> discussion. I tend to think that if we want to do things like testing |
33 |
> permutations and such then automated build-only tools might be the way |
34 |
> to address this. Manual effort should be focused on things like |
35 |
> runtime testing where it adds the most value. This also strikes me as |
36 |
> the sort of thing that could probably be assigned out to volunteers |
37 |
> who do not have commit access. |
38 |
|
39 |
There's a few tools for this out there already. I've personally been |
40 |
working to update app-portage/tatt for git - see |
41 |
https://asciinema.org/a/cqsy983t9jimszvypcjr3zg5m for a demo. |
42 |
|
43 |
> |
44 |
> It really seems like the sort of thing that could be managed by |
45 |
> something other than bugzilla. Some tool finds out about packages |
46 |
> that ought to be stabilized (probably via multiple methods), then it |
47 |
> triggers the automated build tests/etc that do a lot of the low-level |
48 |
> QA, and if the package looks good it gets queued for runtime testing. |
49 |
> Then volunteers report in on status and when whatever criteria we |
50 |
> establish is met then the tool stabilizes the package, probably in a |
51 |
> dependency-aware fashion. Obviously this would require some care for |
52 |
> coordinated packages like xorg/DEs/etc, and it might not be the |
53 |
> preferred approach for many system packages. |
54 |
> |
55 |
|
56 |
We're working on a tool like this in #gentoo-grumpy - new contributors |
57 |
welcome! |