[gentoo-dev] RFC: Portage local package revisions

Hello all,

In designing an enterprise infrastructure around Gentoo at my place of
employment, I have discovered a feature that would improve Gentoo's
usefulness greatly in this field.

I'm writing to ask for your opinion on a change to sys-apps/portage that
would allow users to maintain local revisions of ebuilds, such as
"net-www/apache-2.0.58-r2-local1". This would require a modification to
the ebuild version specifications and a patch to two portage source
files which you can review here:

http://dev.gentoo.org/~malverian/portage_local_version.patch

(Please don't complain about the code quality, I cleaned up areas where
it was desparately needed such as the string.atof() areas, but for the
sake of code coherence, I tried to use the same methodology used elsewhere
in the code as much as possible, such as unqualified except statements :P)

There are a number of scenarios where such a feature is useful, most of
which revolve around the need for a local version bump:

1) You have machines using apache-2.0.58-r2 with an unpatched security
   hole which you would like to immediately patch locally until a fix is
   committed to the portage tree.

2) You are using binary packages and need to simulate a version bump to
   force re-installation of a binary package with modified USE flags.

3) You are using binary packages and need to simulate a version bump to
   force re-installation of binary packages that were rebuilt during
   revdep-rebuild

In all of the above cases, one could simply bump the package up one
revision by creating an ebuild in an overlay for apache-2.0.58-r3.
However, using this solution will result in apache not being upgraded
when apache-2.0.58-r3 is actually committed to the portage tree unless
you perpetuate this bad habit ad nauseum.

To give a better explanation of #3, consider the following scenario:

- You have 60 servers with mysql-4.0.28 and php-5.1.6-r2
- You want to upgrade to mysql-5.0.30 and continue to use php-5.1.6-r2
- You use binary packages which are built on a staging machine
- Your servers know to upgrade via a "pull" method with the help of
  cfengine which tells each server what packages SHOULD be installed on
  it. This also makes it very easy later to build another copy of a
  machine in the case of hardware failure by using the same description
  files (cfengine config programs)

To accomplish the above in an enterprise environment, you would need to
perform the following steps:

1) Install mysql-5.0.30 on the staging machine and build binary packages

2) Perform a revdep-rebuild on all packages using libmysql client
   libraries, building new binary packages for each of them
   
3) Tell your other 60 servers that it is time to upgrade mysql (and in
   this case, reinstall php)

Assuming you have a description file such as the following:

__CUT__

webserver_packages =
(
   dev-db/mysql-4.0.28
   dev-lang/php-5.1.6-r2
)

...

__CUT__

It is obvious what change must be made to install the new version of
MySQL, but how do you update PHP without bumping the version of PHP? You
would need some extra metadata to tell the servers if they are upgrading
from mysql-4.0.28 to mysql-5.0.30 that they should reinstall PHP. This is
fine for a few packages, but it can quickly become a maintanence
nightmare.

Having local revision numbers solves this problem very simply, and
provides quite a bit of flexibility as a side-effect.


NOTES:

- The "-local#" revisions MUST NOT be used in the main portage tree. It
  is something that system administrators would have exclusively for their
  own purposes such as those described above. This SHOULD probably be
  enforced by repoman in addition to policy changes.
  
- The new (completely backward compatible) version priority order would be:

    apache-2.0.58
    apache-2.0.58-r1
    apache-2.0.58-r2
    apache-2.0.58-r2-local1
    apache-2.0.58-r2-local2
    apache-2.0.58-r3
    apache-2.0.59

-- 
gentoo-dev@g.o mailing list