[gentoo-dev] [RFC] New metastructure proposal

[Alexandre Buisse]

Hi everyone,

as everyone probably noticed, there is a current atmosphere of sinking ship,
with quite a lot of people leaving and many agreeing that gentoo is no fun
working on anymore. Before it's too late, I'd like to propose a big reformation
that would help solve some of the issues we are currently having and,
hopefully, bring back some of the fun we all had developing and using this
distribution.


The basic idea is to make gentoo a lot more meta than it already is. Something
that is said quite often is that gentoo lacks a direction. Some people want it
to be a business-oriented distribution, some want it to be bleeding-edge, some
a multimedia, development platform, you name it. Obviously, arbitrarily
choosing one of those directions would mean losing a lot of developers, and
this is something we can't afford to do. The solution, of course, is to go
meta: provide a set of tools that allow people to build the distribution of
their dreams.
This is something that we are already trying to do, but my opinion is that we
are not going far enough. For one thing, we target users as the one who should
build and customize their distribution, while it would be pretty interesting
to also target ourselves as the ones who should be doing this "instantiation"
work. Stage 4's were going in this direction, but they were too isolated and, as
far as I know, they are dead now.



The idea is pretty simple: modularization. There is a core part, with a couple
hundred packages that are absolutely necessary to a system. Then we have a
hierarchy of overlays with additional ebuilds for people's need. Top-level
projects could look like: desktop, dev, business, embedded, misc. Then we
would have subprojects, e.g. multimedia, DE, games for desktop, multimedia
being itself subdivided in audio and video, and so on. This would get us a
tree of arbitrary depth, with development happening mostly in leaves. The
hierarchy would mostly serve as a classification tool, and projects would not
necessarily share resources, including devs, with their subprojects, neither
should they have decision power over them.

This structure has many advantages, one of those being that since projects are
autonomous, they handle their own recruitment, which helps the dev/user
distinction fade away. It concentrates development in small areas where people
will know each other well and be able to interact efficiently.  By
decentralizing, we remove the need for a big authority and give everyone the
freedom they want. The current devrel authority is reduced to only the core
project, though people could still ask its wisdom whenever conflicts pop up.
Basically, the only job involving red tape and a central authority is deciding
who gets the nice "gentoo official project" stamp, and the resources infra can
then provide. Of course, QA would be the main, if not only, criterion in this
choice.

By having everything as modular as possible, we also allow an easy fork of a
single project, for whatever reason. So if enough people think that mozilla is
being badly maintained by the current project and that people in it don't want
or can't apply their fixes, they can easily provide their own overlay with
better ebuilds.  And if their version is indeed better, over time it will get
the official status and have superseeded smoothly the first project. Think of
paludis and pkgcore vs portage.



So far, I've come up with two main disadvantages to this reformation. The first
is that dependencies between different projects could become a problem if not
handled carefully. For one thing, they would require a change to ebuild
syntax, along the lines of DEPEND="dev.perl.libs:libfoo", and of course
support from package managers. Pulling single ebuilds when required instead of
a full overlay would be a nice thing to have as well. Another idea to simplify
this would be to have a central DB with every known ebuild in it (including
non-official, bad QA ones) and the names of repositories/projects providing
it. This would give enough information to package managers for them to make
intelligent choices about how they should behave.

The other big problem is that a migration to this structure would require a
lot of effort from everyone. The good news is that we could use the
opportunity to migrate to a nicer scm (and given what gentoo would look like,
a distributed one like mercurial or git would be a natural choice) and also
that we would get a good idea of what is maintained and what isn't. Leaving
out stuff that no-one wants would then be very easy. Also, I believe that by
having a clear goal, everyone should get a huge motivation boost and I believe
that things could go quite smoothly, and even quickly.



Of course, many details have been left out that should be discussed and solved
before any decision is taken. Among those is the status of arch teams, which
is a bit unclear. An idea would be to have the main three or four most-used
arches (x86, amd64, ppc would be my guess) in KEYWORDS, as usual, along with a
list of repositories that given person is allowed to keyword in, keeping arch
teams organized as they currently are. Other arches would be projects of their
own, with some kind of meta-overlay that specifies which ebuild from which
overlay has been tested, etc. Or we could make no distinction between popular
and less popular arches, I don't really have any opinion on the matter. 
Also, what to do about stuff that isn't specific to any project, even
though it wouldn't happen so often? For instance, deciding whether we
should participate in SoC, or this kind of things. We could use a
council as currently, or ask for representatives of all official
projects to punctually decide, or organize a general vote, or maybe even
something else.



To end this proposal, let me say that without a doubt, there are many holes
and hidden problems in it. I don't claim it's perfect, nor that it will
magically solve our problems. But I think it is a better structure than the
one we currently have, and that switching would reduce, perhaps even stop, the
dev bleeding, bring back freedom as well as fun, and help ease the tensions.

Please criticize this with everything constructive you can think of.


Thanks,
/Alexandre
-- 
Hi, I'm a .signature virus! Please copy me in your ~/.signature.