1 |
On wto, 2017-07-25 at 22:59 +1000, Michael Palimaka wrote: |
2 |
> On 07/25/2017 07:22 AM, Sergei Trofimovich wrote: |
3 |
> > 2. Q: How to make arch testing faster and easier? |
4 |
> > |
5 |
> > A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing |
6 |
> > required" will be automatically tested and keyworded. |
7 |
> > |
8 |
> > [handwave] automated tinderbox setup would help a lot |
9 |
> > to now upfront what fails to built, fails tests. |
10 |
> |
11 |
> I've had similar thoughts about this and have already been working on |
12 |
> some tooling for this. |
13 |
> |
14 |
> We would need to establish exactly what criteria must be met for an |
15 |
> automated test to be considered as successful. |
16 |
> |
17 |
> Here's a sample report that my tool produces: |
18 |
> https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html |
19 |
> |
20 |
> In this case, would it be enough that it builds and tests pass? What |
21 |
> about the QA issues? Do we need someone to review them to determine if |
22 |
> they should block stabilisation, or if they're even a regression or not? |
23 |
> |
24 |
|
25 |
There are different kinds of QA issues and they have different |
26 |
significance. You can ignore some of them, some others are more |
27 |
important. |
28 |
|
29 |
GLEP 65 defines a 'eqatag' function that the checks can use to provide |
30 |
machine-readable results for the checks. You should work with that |
31 |
instead of parsing the verbose messages. |
32 |
|
33 |
-- |
34 |
Best regards, |
35 |
Michał Górny |