Re: [gentoo-dev] QA Roles v2

[Patrick Lauer]

On Thu, 2006-03-02 at 09:01 +0000, Stuart Herbert wrote:
[snip]
> * There's nothing in this policy about end users.  If this QA team is
> not *focused* on delivering benefit to end users, then (as has
> happened this week) it becomes a self-serving team, focused instead on
> what can only be described as a destructive path.  No-one benefits
> from that, no-one at all.
>
> * The QA team is asking for more than it needs to perform its role.
> The UNIX principle is that of "least privilege".  Donnie's already
> pointed out that FreeBSD is able to conduct effective QA without the
> extra power that the QA team is continuing to ask for.
So I see two scenarios for that:
- A QA team with a purely advisory function, helping with communication.
pro: no big policy changes etc.
con: teethless QA may get ignored

- A QA team with limited executive power, fixing bugs as they are found
pro: fast turnaround times on bugs
con: resistance from developers

The second approach needs to be carefully implemented, people need to
have trust in the QA team to empower them.

> * There is no proposal for a process to formulate, and gain wide
> approval for new QA standards.  This week, there's been an example of
> the QA team documenting a QA standard *after* a bug was raised about a
> QA violation ... and then that document being used as if that
> particular QA standard had always been in the document.
Communications issue. This thread should help fix the policies for that I hope.

> Mistakes will always be made by all developers, and good QA is
> essential to Gentoo's future.  We need a good QA team for Gentoo.  Not
> having a QA team is, in my eyes, not an option at all.
Fully agreed.
> But, as this week has shown, QA members are also developers (and
> human), and are just as capable of making mistakes as anyone else.
Obviously :-)
> We need a quality assurance team that conducts all its activities in a
> quality manner.  I'm not just talking about personal behaviour, or of
> any one individual.  The way *everything* is done must be in a quality
> manner.  That should mean a high quality process for creating QA
> standards, having them approved, and making sure developers know what
> changes are coming and when.  That should mean high quality automated
> tools that cope with the real world, not some ivory tower that has no
> real pay-off for users.  It should mean an interpretation and
> application of QA standards that is focused on how it improves matters
> for real users - and not a "tick in a box" QA approach.  It should
> mean a team of educators, not a team out to bully with the mandate to
> do so.
That sounds like a mission statement and should be part of QA policy

> In twelve years of being a professional software engineer, I've never
> seen a successful QA team that didn't match that description above.
>
> Mark, in the discussions about the QA policy, your fallback
> justification always seems to be "Trust us".  I think this week's
> events have put a big dent in the credibility of that argument, if not
> holed it below the water line.  If the QA team followed processes
> similar to what I've described above, I believe that this week's
> events wouldn't have happened.  What started off as a worthy piece of
> QA work, which I'm sure has fixed many real problems for users,
> degenerated into something altogether unpleasant and unnecessary for
> all involved.  We've all gotten a week older and a week greyer out of
> this.  Have we fixed any real problems that stop our users installing
> and running Gentoo?  No, we haven't.  I hope we can all (and I include
> myself in that) learn something from this to prevent a repeat.
>
> I call for Mark's proposed policy to be rejected as it stands.
I'd like to see it extended with the ideas shown in this thread. Also
the QA team should consider ways of getting higher acceptance - I
suggest that a general vote should be done, that's about as democratic
as we can get and noone can weasel put after that (although I'm open for
other processes to give the QA team support)

Patrick 
-- 
Stand still, and let the rest of the universe move