Re: [gentoo-dev] [RFC] QA Team's role

[Stuart Herbert]

Hi Mark,

On 2/27/06, Mark Loeser <halcy0n@g.o> wrote:
> Your change seems to imply that the QA team must wait for the council's
> okay to go forth and fix the package, rather the QA team able to act on
> its own.  If that is the case, I don't see how we would ever be able to
> get things done when someone disagrees with us.

In the event of a disagreement, that's exactly what I'm asking for. 
Hopefully, disagreements will be rare.  But, when they do arise, and
it is not an emergency, I see no reason why the QA team needs the
ability to force its point of view onto others.

> Again, then we are going to get into the argument of the definition of
> an emergency and never be able to get anything done.  We really hope
> problems never come down to this, which is why we left it worded this
> way.

Me too.  But it will, sooner or later, and when something isn't an
emergency, there's no reason why a change cannot wait until the
dispute has been resolved.

I have no desire to stop the QA team being able to act in an
emergency.  It's in all our interests for action to be taken in an
emergency.

> > The QA team has no monopoly on what is right or wrong in Gentoo, and
> > neither do package maintainers.  If there is a disagreement that leads
> > to an appeal, the onus should be on the QA team to have to present their
> > case to the council, as they are the ones pushing for change.
>
> Again, this is taking away any power that the QA team might have

I find that an interesting statement.  The only power my proposals
remove is to stop you bullying people by insisting you are always
right before a peer review has happened.  If there is a genuine QA
problem, and you can't convince the developer in question of that,
there's still a fair process that allows you to enforce when concensus
has failed.

Without these safeguards, my feeling is that the QA team is free to
enforce without concensus at all times.  I don't believe that is in
the best interests of Gentoo, and is a significant shift in the way we
govern ourselves.

I don't see any compelling reason for the QA team to be judge, jury
and executioner, with the council acting as a post-execution appeals
panel.  Wasn't devrel broken up into separate investigation and
enforcement teams over these very same concerns, raised by a member of
the QA team?

> This is not quantifiable at all and will just get us into arguments with
> people that disagree with us that there is a problem present.

If you mean that it creates grey areas where the output of automated
tools cannot always be right, I agree.  If you're saying that it's
beyond your capacity as human beings to weigh up the merits of an
argument on more than just narrowly-defined technical facts, then are
you best placed to be making the decision in the first place?

If a policy is to be supportable and implementable, it has to be
reasonable, and it has to be managed by reasonable people.  I feel
that what you're asking for, on this point, is more dogmatic than
reasonable.

Case in point.  I've presented to you that, after two and a half
years, the situation that has sparked this debate off hasn't affected
users.  I've explained that it is a third scenario, and that it is
different to the two (legitimate) scenarios that you've been busy
getting fixed.  From where I'm sat, it would seem reasonable to accept
that, although this is a problem when looked at purely from a
technical point of view, this scenario isn't causing problems at this
time, and we could all move on to dealing with more important matters.
 If there was a real problem, there would be enough evidence after two
and a half years for you to show me, and that would convince me (and
any other reasonable person) that I was wrong, and that action was
worth taking.

You haven't presented that evidence, or if you have, I haven't seen it
since getting up this morning.

Instead, we have a proposed QA policy that says "we're always right,
and when we're not, we still get our way until you convince the
council to let you back out the changes we demand."  I think that's
unreasonable.  That's why I oppose this point.  To me, it smacks of
wanting to have your own way whether you're right or not.  I can't
support that.

> Again, this bogs us down in useless process if someone wants to argue a
> change that clearly breaks things, but isn't documented.  It is
> impossible to document every single thing that is a problem, and we are
> asking for some freedom to be able to work on issues that fall under QA.

As already mentioned, if it's not documented, how on earth do you
expect to raise the general quality of the QA done by each and every
developer?  How do you expect to be able to consistently enforce your
own QA standards?

If it's not documented, then you're left with making it up as you go
along.  That's in no-one's interest.

This comes back to my point about the QA team needing to be an
educational role first and foremost.

> So the Portage team will have to agree with us on every single problem
> that is a QA issue?  This seems like something we can leave alone until
> it doesn't work, at which point we bring it before the council to figure
> out how we can best handle this situation.  Saying that another team
> must approve all QA checks just seems silly.

I'm asserting that it isn't working.  Case in point.  Brian, as
co-lead of the Portage team, has said that we won't be adding
DEST_PREFIX to Portage.  If the Portage team won't agree with your
interpretation of what is or isn't a valid QA check, why should you
have the right to go ahead anyway and enforce that check on the
package maintainers?

It's not silly.  What do you have to fear about having your proposed
QA standards backed by key teams?  If your arguments have merit, they
will be supported.

What I think is silly is the QA team claiming for itself the power to
enforce standards just on their say so.

> All of this seems to assume that the QA team is going to abuse its
> power.  So far we have had very few problems when we ask people to
> fix issues that we have found for their packages.  Is this going to
> always be true?  No, and someone needs to be able to get problems fixed
> instead of just leaving things the way they are, and we want to fill
> that role.

I think you're already abusing that power, by calling for a package to
be removed when it's causing no trouble to anyone, and when the
workarounds create a worse user experience than what is already there.
 When the developer in question declines to remove the package, your
response is a proposed QA mandate that is unbalanced.

Genuine problems need to be fixed.  My proposals do not stop you from
doing that.  They also ensure that we have a balanced QA approach that
genuinely serves the needs of the project.

> > I'd like to see the QA team's primary role stated as "educational"
> > first, then watchdog, then intervener in that order (emergencies
> > not-withstanding).
>
> Which is what we plan on doing, and have been doing so far.

Walk the talk.  Create the educational material.

> We are working on a document, and hope to document all of the problems
> and why they are problems.  We don't plan on saying something is just
> wrong and not give an explanation.

:)

Best regards,
Stu
--

-- 
gentoo-dev@g.o mailing list