[gentoo-server] Random thoughts about portage maintenance

[Jose Gonzalez Gomez]

I've been thinking about the problem stated a few days ago about portage 
maintenance... first of all, I'm not on the developer list, so I don'tknow if 
many of the assumptions I'm making here are right or wrong, so a big "correct 
me if I'm wrong" must be assumed everywhere... well, here it goes...

THE PROBLEM

Portage is slowly getting out of date. Some facts: portage has 7600 packages, 
and more than 80.000 files, but only 200 devs. This is clearly insufficient 
to maintain such a system.

DEVS RESPONSABILITIES

The Gentoo devs have the following responsabilities (correct me if I'm wrong):
- Mantain a structure giving support for Gentoo
- Develop portage (the core of Gentoo)
- Enhance the current platform with the creation of new projects specific to 
Gentoo
- Create and mantain ebuilds
- Create and mantain Gentoo specific patches for those ebuilds
- Resolve bugs created at bugs.gentoo.org

MY RANDOM THOUGHTS

A. REGISTERED USERS / TESTERS
Well, first of all, open source software is all about participation, but how 
many Gentoo users would want to become a developer? Probably few. We all have 
college, works, families, hobbies,... and Gentoo would be another thing 
stealing our precious time, so how to get more developers? Does Gentoo really 
need more developers?

Let's see... from my point of view, there are "core" responsabilities and 
"secondary" responsabilities. I mean that the real heart of Gentoo is 
basically portage, so I think core developers should concentrate on further 
developing and enhancing the Gentoo base, while leaving the "mundane" tasks 
of creating ebuilds and resolving bugs to other people, or maybe a second 
level of developers. Anyway we still have the problem of few developers 
dealing with a lot of ebuilds.

I think (again correct me if I'm wrong) that a great part of the ebuild 
creation and maintenance task consist of testing the ebuilds, so they install 
the software properly, and there are no problems once that software is 
installed. Then that ebuild enters the unstable branch, and if there are no 
further problems, the ebuild become stable... and here is where I think the 
system may be greatly improved:

Let's say Gentoo creates new posts: registered users and/or registered 
testers. Ok, maybe there are few out there wanting to become a developer, but 
there are tons of users out there continously installing and testing Gentoo. 
Why don't take advantage of that? The developer task may take a lot of time, 
and may be not rewarded, but testing new software is something that people 
uses to do while they're studying, working, or even playing at home. Let's 
make them take a step further: become a registered user or tester that 
reports directly to Gentoo their success or failure, instead of just writing 
to some list saying they have a problem, or they've been successful while 
installing and using an ebuild. 

I would do this per package. I mean a registered user/tester would be 
associated with a set of packages she uses on a daily basis. Testers would 
have an associated rating of reliability (I don't know if this is the best 
word for this) and would be notified whenever a new version of the package 
comes out. So I see the process somewhat like this:

1. Somebody (dev or not) writes a new ebuild
2. If not submitted by a dev, a dev checks the ebuild for form defects (bad 
CVS tags, USE flags unproperly used,...) and admits or rejects the ebuild
3. Registered testers are notified of it
4. During testing period, registered testers report success or failure, 
weighed up based on their reliability rating. The ebuild may change, 
resetting the testing period.
5. Once the testing period has finished, if the average of success reports 
weighed up with reliability ratings is greater that a threshold, accept the 
ebuild and put it in the unstable branch, maybe repeating the process with 
registered users.

I guess this is what is done right now at bugzilla in an informal way, but I 
think that formalizing this process would have a few benefits:

1. Right now (at least in my case) people know of new ebuilds in bugzilla when 
they actively search for a package in bugzilla. The automatic notify process 
would greatly increase the testing of new ebuilds just created in bugzilla, 
instead of limiting this testing to people that just has found the bug
2. A tester has a much lower responsability and time dedication than a 
developer, so people in their daily jobs could develop this role
3. The testing work of a developer could be delegated on these testers, and in 
turn they could use their time for more important things, just having the 
responsability of reviewing reports and taking a decission based on that.

One of these testers could eventually become an ebuild submitter, or even a 
package mantainer.

B. AUTOMATIC INFORMATION RETRIEVAL
Another thing that I think could be interesting is some form of automated 
sending of information about installed packages in test/production 
environments. Registered users / testers could install a tool in their 
systems that would send information to some kind of server about their 
platform, gcc/glibc versions and options, use flags, installed packages, ... 
That information would then be used by some data mining techniques to find 
information about bugs, taking into account reports from the tester, so the 
system could relate success/failure reports with the state of the system in a 
particular moment. Maybe even this tool could also be used to send the 
reports, so the information is included in it.


Well, that's all by now, best regards
Jose