[gentoo-dev] [RFC] gentoo-x86 migration to repo-per-package

[Fabian Groffen]

All,

When we migrate away from CVS for gentoo-x86 (gx86), as it looks now,
the same structure will be kept as we have in CVS now.  Policies to
reject merge commits and only allow rebases on e.g. the Git
infrastructure will even more closely match the central and server-based
way of working Gentoo is used to now.

In this email, I step away from the current model that Gentoo uses for
the gentoo-x86 repository.  Instead, I consider a repo-per-package
model, as in use by e.g. Fedora [1] and Debian [2].

In short, the repo-per-package model means that each package
(my-cat/package) is a separate repository in some VCS.
Instead of having a huge tree that will only grow forever (gx86),
packages are just in their own repository.

This approach can potentially be interesting for a number of reasons:
- history is per package
  + package is likely to be small enough not to have to consider any
    history removal/splitting -- everything can be retained
  + if a package is removed, it's repository is simply no longer
    considered, hence its existence and history doesn't clobber a main
    repository
  + since the repository can move, its history can also easily move
    along with its location, being either a category, or even as purpose
    (e.g. packages that started on sunrise, or in developer overlays)
- tree generation is dynamic
  + a full (rsync) tree has to be created by first getting all repositories,
    and/or getting them up-to-date -- avoid those packages you don't
    need
  + easy to make different "trees", e.g. a server tree (no GNOME, KDE),
    prefix tree (different versions of packages), etc.
  + easy to move packages around, their category is specified by the
    tree configuration, the repository the package lives in doesn't change,
    probably overlays, betagarden, graveyard, sunset, etc. can all go
  + no restriction to using only a single VCS, because packages are just
	refreshed before inclusion in the tree, their (source) origin
    doesn't matter
- per package branches
  + instead of developing in overlays, simply branches could be used,
    such that a single place is sufficient to for each package
  + switching branches can implement atomic tree-wide changes for
    complex situations

No restriction to using a only a single VCS.
While I don't think that allowing developers to use their VCS of choice
is very relevant when committing package changes, the ability to use
more than just *one* VCS when assembling the rsync tree is a huge
advantage if we want to migrate away from the current CVS tree slowly
during a migration period.  It could enable the use of git (the obvious
choice of many) now, alongside the current gx86 tree.

Because the rsync tree would be generated by assembling all packages
that need to be in the tree, the only thing necessary for that
generation is to understand which VCS commands to use to acquire/update
a package and what files/directories to skip when copying the package to
its final destination in the rsync tree.  This is easily scriptable,
given that only the old gx86 tree will be CVS, and the rest git.

When rsync tree generation would be based on a file with packages to
include, I can imagine a simple way define where the package comes from,
and where it should end up, e.g.:

  my-cat;package1;git://git.g.o/foo-package.git;optionalbranch
  my-cat;package2;cvs://cvs.g.o/gentoo-x86/my-cat/package2;

which defines the category, the package name, its source location, and
perhaps something like a branch or tag in case we ever want to e.g.
split development (what is now in overlays typically) from what's in the
tree.  Branch support would also be useful for e.g. Prefix modifications
to a package, when only checked out by Prefix rsync tree generation.  It
can as well be a solution for what is often referred to as the "slacker
arches" problem, when old versions of ebuilds that a maintainer wants to
drop, remain available for minority arches that need them, but only for
their rsync tree, without bothering the maintainer with it.

Obviously, in an ideal world all packages would be in the same VCS, git
in this case.  With a system like this, however, CVS packages can slowly
be moved to git, as their maintainers see fit.  Some developers aim to
benefit more from git than others.  They can move their packages,
directly.  For the remaining packages, eventual migration is necessary,
but they should block developers that want to use git for their packages
less.

There probably are drawbacks to this system as well.  I, however, only
see big advantages for the moment.
Comments, thoughts, ideas welcome.


[1] http://pkgs.fedoraproject.org/gitweb/
[2] http://packages.qa.debian.org/common/index.html

-- 
Fabian Groffen
Gentoo on a different level