Re: [gentoo-osx] Arch Testing Policy and Procedures

[Lina Pezzella]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Sep 4, 2005, at 6:57 AM, Grobian wrote:

> Although I like the arch testers (ATs) idea pretty much, I have
> some questions:
>
> - Why have the 'low hanging fruit' bugs from before 2005 not been
>   resolved till I run through them a few weeks ago?.  Most of them
> were
>   small and had no USE flags.  I expect the devs that had no time
> to do
>   this in the past won't have time to do it when an AT has run over it

This is exactly the point of arch testers. The idea is that AT's will  
evaluate those bugs and the developer will trust the AT's opinion and  
commit without redoing their work. It does not take much time to  
keyword and commit a package. The time drain is in compiling and  
testing the package.

>   In the end, the dev is responsible for the commit, not the AT,
> hence I
>   would not be surprised if the dev does a (non USE flag extensive)
>   compilation on his own machine before committing to CVS.

This is probably a good idea until a trust relationship between the  
dev and AT is established. For example, I rechecked most of your work  
as a developer candidate in the beginning, but after a week or so I  
stopped checking and committed what you indicated on trust. I would  
expect the same to happen with the new AT team.

> A trust relationship between dev and AT is neccessary too and needs
> to be
>   built.

Absolutely. This will come from the AT doing good work and devs being  
prompt about responding to an AT's work. If each developer dedicated  
just an hour each week to going through AT work and committing what  
was indicated, I think it would be fairly easy to keep up with the  
load. You can do a large number of commits in an hour.

> - Maybe AT's should be 'assigned' to one or two devs who are
>   responsible for committing the AT's work.  This is a natural mentor
>   relationship as well as QA wise, it is obvious who is going to
> punish
>   you if your work is not correct ;)

I agree that we should have some sort of probation set up for AT's  
that do not do good work. I don't want to 'assign' AT's to devs as  
that implies that a dev is overseeing the AT's work. In order for  
AT's to help reduce our workload as devs they need to be a trusted/ 
independent group. Developers are not babysitters.

> - This proposal assumes ATs have time, which devs apparently have not.

Not true. If I didn't have time, I wouldn't be doing Gentoo. I choose  
to spend my time as a developer on larger projects then simply  
testing and keywording ebuilds.

>   I agree that the AT work is simple, but boring.  Though I still like
>   to know why it hasn't been done in the past.

It has been done by other archs and is recently becoming more  
mainstream within Gentoo.

> - Assuming I'd have an AT or two assigned to me,

That's not the way I'd like this to work.

> I'd like to have the
>   freedom to give them more flesh if they are hungry for it, i.e. dive
>   into why something doesn't configure/build/compile/install and
> try to
>   come up with a patch.

Having AT's gives us a good way to evaluate potential new developers.  
My hope is that if an AT does good work and is interested in becoming  
a Gentoo developer, we can give them "more flesh", as you put it. It  
should be clear though that "more flesh" is not the responsibility of  
an AT.

> Maybe this is dev dependant, but I think for an
>   AT it would be nice to know there is a road upstairs: if they're
> good,
>   and do what they do very well, it would be nice if I wouldn't
> have to
>   commit their work.  (in other words: promote such AT to a dev)
> On the
>   other hand, you still need the AT work to be done.

Yes, promote the existing AT to a developer and get a new AT. Perhaps  
have the old AT mentor the new AT. These are all things to be decided  
upon as part of our policy on becoming a Gentoo for Mac OS X  
developer (which I think should also be written out instead of being  
unspoken). I feel that this is a subject for another policy page and  
another e-mail. For now, I'd like to focus on getting a group of AT's  
in place.

> I think the draft is a good piece of work.  I'm almost eager to
> have one, like a PhD who gets an MSc assigned to him.  My
> experiences with some bug reporters who were also in IRC to fix
> bugs using direct feedback from them is very productive, however if
> I slam myself in the face and throw some cold water over my head I
> feel myself forced to look at the issue from a much more
> pessimistic point of view considering this team and the current
> 'productiveness'.

Don't assume that not seeing work from a developer means they're not  
working. For instance, as a result of recent emails on this list,  
Hasan and I have moved a large chunk of our time that was previously  
spent on porting/keywording e-mails to drafting documentation.  
Additionally, a lot of developers choose to work on long-term  
projects in an overlay and thus don't necessarily commit things to  
the portage tree proper on a regular basis.

This being said, there are a good number of inactive developers. In a  
previous e-mail on this thread, Hasan and I explained our inability  
to take care of this problem without assuming the "Lead" position.  
Since there were no direct objections to our taking this position, we  
will make the appropriate changes to the project page[1] and begin  
conversation with devrel regarding our inactive developers. Hopefully  
this is still okay with everyone. More to follow regarding this in a  
new thread once we have more information.

> Currently, we only discuss the way 'up', but maybe the way 'down'
> should be in the picture too.  An AT should be considered to be
> 'active'. Where active means that such AT can do some useful work
> on a regular base.  (Note: this implicitly requires devs to be at
> least as active as the AT.)  If not, while there is work enough,
> such AT should be removed.  Might sound obvious, but if there are
> no hard rules for it, noone will get removed.

Agreed. So far, then, we need documentation/policy on the following:

1) Inactivity (Developers and ATs)
2) Probation for ATs (Probation for Devs is covered by devrel)
3) Selection of New Developers / New ATs
4) What each developer is working on (10,000ft overview)
5) Specific/Misc Policy such as where to install python modules,  
package.mask usage, Unstable/Stable Keywording, etc

All of this will be in the form of a Gentoo for Mac OS X policy  
handbook, which will take time (specifically (5) above). Suggestions  
for what should go in here would be highly valued.

> Yes, nice idea, but I think we should look at the problem from
> inside this very team first.  I would consider the average
> participation level very unhealthy.

Again, please don't confuse lack of commits with lack of  
participation. Also note that we have only a handful of active  
developers. Once we agree upon policy for inactivity, we can make  
some progress with weeding out those that are not contributing.

> This team is also very opaque, it's almost impossible to know what
> someone is working on.

Like we said, we'll address this shortly. We felt that AT's are more  
important than this right now. To give an idea:

Kito is working on Darwin stuff. I'm sure he can elaborate if he  
cares to.
Hasan is working on autotools and baselayout design issues for the  
collision-protect
profiles as well as ipv6 support for qt.
I am just completing my work with adding dylib support to ffmpeg. I  
also take care of sci-biology for ppc-macos. Next project is fixing  
esound.

And of course, Hasan and I are both working on drafting documentation/ 
policy such as the AT Policy/Procedures draft we just sent out. If  
there is more documentation/policy that you feel is necessary, please  
start another thread and let all of us know. It's hard for us to tell  
what is needed since we've been here a while.

Before discussing these other issues, however, can we all agree upon  
what is necessary to bring ATs on board? Is the documentation we have  
sufficient for that purpose, or is there more that needs to be added?  
(Note: policy on converting ATs to Devs is not needed yet.)

- --Lina Pezzella
Ebuild & Porting Co-Lead
Gentoo for OS X

[1] http://www.gentoo.org/proj/en/gentoo-alt/macos/index.xml

>
> Lina Pezzella wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> Our apologies for the tardiness of this e-mail; we have been
>> preoccupied with moving back to college this last week.
>> We have thrown together some draft documentation[1] regarding our
>> expectations for ppc-macos arch testers. We have purposely
>> neglected to include policy and procedures for dealing with stable
>> keywords pending the current reevaluation of our decision to
>> maintain them. For all interested, please review the document and
>> tell us what you think. The faster we can all agree on policy and
>> procedures, the faster we can get arch testers on board.
>> For those that missed the initial arch tester discussion, the hope
>> is that having a dedicated group of arch testers will improve QA
>> as well as free up developers to solve design and porting issues
>> rather than keyword requests and package testing. It is also a
>> great place for potential new developers to gain experience with
>> the project.
>> More policy and procedure documentation to come.
>> - --Lina Pezzella && Hasan Khalil
>> Ebuild & Porting Co-Leads
>> Gentoo for OS X
>> [1] http://dev.gentoo.org/~gongloo/macos/doc/at-procedures.html
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.2 (Darwin)
>> iD8DBQFDGlv7NJ9STR9DbYERAqJ1AKCgQ73vaFfulp1tvXt3FhMOAckZvgCgqO9t
>> +xaXd/DKXUW0ZmJxomn8vYw=
>> =GZ6D
>> -----END PGP SIGNATURE-----
>> --gentoo-osx@g.o mailing list
>>
>
> --
> Fabian Groffen
> Gentoo for Mac OS X
> --
> gentoo-osx@g.o mailing list
>
>


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDG0lZNJ9STR9DbYERAh9XAKDgy4xA270naCwyzOnDOC8a/5cC2ACbBHnx
FyQ6uGgz2kUN21unshZFxyM=
=YPW3
-----END PGP SIGNATURE-----
-- 
gentoo-osx@g.o mailing list