Re: [gentoo-dev] QA Roles v2

[Stuart Herbert]

Hi Mark,

This draft seems to be effectively the same as the last one.

On 3/2/06, Mark Loeser <halcy0n@g.o> wrote:
> * The QA team's purpose is to provide cross-team assistance in keeping
>  the tree in a good state. This is done primarily by finding and pointing
>  out issues to maintainers and, where necessary, taking direct action.

Same as original version.

> * In case of emergency, or if package maintainers refuse to cooperate,
>  the QA team may take action themselves to fix the problem.

Same as original version.

> * The QA team may also offer to fix obvious typos and similar minor
>  issues, and silence from the package maintainers can be taken as agreement in
>  such situations.

Same as original version.

> * In the event that a developer still insists that a package does not
>  break QA standards, an appeal can be made at the next council meeting. The
>  package should be dealt with per QA's request until such a time that a
>  decision is made by the council.

Same as original version.

> * In the case of disagreement on policy among QA members, the majority
>  of established QA members must agree with the action.

Same as original version.

> * Just because a particular QA violation has yet to cause an issue does
>  not change the fact that it is still a QA violation.

Same as original version.

> * If a particular developer persistently causes breakage, the QA team
>  may request that devrel re-evaluates that developer's commit rights.
>  Evidence of past breakages will be presented with this request to
>  devrel.

Same as original version.

> * The QA team will maintain a list of current "QA Standards" with
>  explanations as to why they are problems, and how to fix the problem.  The
>  list is not meant by any means to be a comprehensive document, but rather a
>  dynamic document that will be updated as new problems are discovered.  The QA
>  team will also do their best to ensure all developer tools are in line with
>  the current QA standards.

The bit about "explanations as to why they are prblems, and how to fix
the problem" is new, as is the statement to ensure that all developer
tools are in line with the current QA standards, but otherwise this is
also the same.

I'm sorry, but personally I don't see how this draft is substantially
different from the one posted originally.  It looks like you've
decided not to address the points I raised about your original draft:

* There's nothing in this policy about end users.  If this QA team is
not *focused* on delivering benefit to end users, then (as has
happened this week) it becomes a self-serving team, focused instead on
what can only be described as a destructive path.  No-one benefits
from that, no-one at all.

* The QA team is asking for more than it needs to perform its role. 
The UNIX principle is that of "least privilege".  Donnie's already
pointed out that FreeBSD is able to conduct effective QA without the
extra power that the QA team is continuing to ask for.

* There is no proposal for a process to formulate, and gain wide
approval for new QA standards.  This week, there's been an example of
the QA team documenting a QA standard *after* a bug was raised about a
QA violation ... and then that document being used as if that
particular QA standard had always been in the document.

Mistakes will always be made by all developers, and good QA is
essential to Gentoo's future.  We need a good QA team for Gentoo.  Not
having a QA team is, in my eyes, not an option at all.

But, as this week has shown, QA members are also developers (and
human), and are just as capable of making mistakes as anyone else.

We need a quality assurance team that conducts all its activities in a
quality manner.  I'm not just talking about personal behaviour, or of
any one individual.  The way *everything* is done must be in a quality
manner.  That should mean a high quality process for creating QA
standards, having them approved, and making sure developers know what
changes are coming and when.  That should mean high quality automated
tools that cope with the real world, not some ivory tower that has no
real pay-off for users.  It should mean an interpretation and
application of QA standards that is focused on how it improves matters
for real users - and not a "tick in a box" QA approach.  It should
mean a team of educators, not a team out to bully with the mandate to
do so.

In twelve years of being a professional software engineer, I've never
seen a successful QA team that didn't match that description above.

Mark, in the discussions about the QA policy, your fallback
justification always seems to be "Trust us".  I think this week's
events have put a big dent in the credibility of that argument, if not
holed it below the water line.  If the QA team followed processes
similar to what I've described above, I believe that this week's
events wouldn't have happened.  What started off as a worthy piece of
QA work, which I'm sure has fixed many real problems for users,
degenerated into something altogether unpleasant and unnecessary for
all involved.  We've all gotten a week older and a week greyer out of
this.  Have we fixed any real problems that stop our users installing
and running Gentoo?  No, we haven't.  I hope we can all (and I include
myself in that) learn something from this to prevent a repeat.

I call for Mark's proposed policy to be rejected as it stands.

Best regards,
Stu

-- 
gentoo-dev@g.o mailing list