1 |
On Thu, Nov 17, 2016 at 2:16 AM, Michael Palimaka <kensington@g.o> wrote: |
2 |
> |
3 |
> In cases where all USE flags combinations are not being tested, it is |
4 |
> still recommended to test: |
5 |
> * with all USE flags enabled |
6 |
> * with all USE flags disabled |
7 |
> * the default USE flag settings |
8 |
> |
9 |
|
10 |
I imagine that in practice only the last of these really tends to get tested. |
11 |
|
12 |
> * A leaf package such as {{package|kde-apps/kcalc}} may not require any |
13 |
> runtime testing at all |
14 |
|
15 |
I'm not really a big fan of this, but if we REALLY didn't want to do |
16 |
any runtime testing on a package then we should have some way to tag |
17 |
the package as such, and then have some way to do automated |
18 |
build-test-only stabilization. If you aren't doing runtime testing |
19 |
then manual stabilization adds zero value. |
20 |
|
21 |
Overall though the writeup was good and maybe it will trigger some |
22 |
discussion. I tend to think that if we want to do things like testing |
23 |
permutations and such then automated build-only tools might be the way |
24 |
to address this. Manual effort should be focused on things like |
25 |
runtime testing where it adds the most value. This also strikes me as |
26 |
the sort of thing that could probably be assigned out to volunteers |
27 |
who do not have commit access. |
28 |
|
29 |
It really seems like the sort of thing that could be managed by |
30 |
something other than bugzilla. Some tool finds out about packages |
31 |
that ought to be stabilized (probably via multiple methods), then it |
32 |
triggers the automated build tests/etc that do a lot of the low-level |
33 |
QA, and if the package looks good it gets queued for runtime testing. |
34 |
Then volunteers report in on status and when whatever criteria we |
35 |
establish is met then the tool stabilizes the package, probably in a |
36 |
dependency-aware fashion. Obviously this would require some care for |
37 |
coordinated packages like xorg/DEs/etc, and it might not be the |
38 |
preferred approach for many system packages. |
39 |
|
40 |
-- |
41 |
Rich |