1 |
El mar, 25-07-2017 a las 22:59 +1000, Michael Palimaka escribió: |
2 |
> On 07/25/2017 07:22 AM, Sergei Trofimovich wrote: |
3 |
> > 2. Q: How to make arch testing faster and easier? |
4 |
> > |
5 |
> > A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing |
6 |
> > required" will be automatically tested and keyworded. |
7 |
> > |
8 |
> > [handwave] automated tinderbox setup would help a lot |
9 |
> > to now upfront what fails to built, fails tests. |
10 |
> |
11 |
> I've had similar thoughts about this and have already been working on |
12 |
> some tooling for this. |
13 |
> |
14 |
> We would need to establish exactly what criteria must be met for an |
15 |
> automated test to be considered as successful. |
16 |
> |
17 |
> Here's a sample report that my tool produces: |
18 |
> https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df01 |
19 |
> 7e14-bd68-47e2-9738-554e7ba1cf10.html |
20 |
> |
21 |
> In this case, would it be enough that it builds and tests pass? What |
22 |
> about the QA issues? |
23 |
|
24 |
Personally, I don't feel QA issues as major enough to block a stabilization, |
25 |
usually they won't cause major issues for stable users and, if they do, for sure |
26 |
they shouldn't be only QA issues :/ |
27 |
|
28 |
> Do we need someone to review them to determine if |
29 |
> they should block stabilisation, or if they're even a regression or not? |
30 |
> |
31 |
|
32 |
Regarding general regressions, that is probably the harder point to handle |
33 |
automatically. In the past, the scripts in arch-tools.git were avoiding to open |
34 |
automated stable bug reports for packages having opened bugs (excluding |
35 |
enhancement and QA bug reports), but that approach is not too good as, for |
36 |
example, having a bug report asking for a version bump but tagged as "normal" |
37 |
and not "enhancement" will lead to the bug not being opened :S |
38 |
|
39 |
Then, I am unsure about if that part can be done automatically :/ |