[gentoo-dev] Portage 1.5_pre Update

Hello all,

Right now, there's a new version of Portage (1.5_pre1) on CVS that features
many major improvements (several that were suggested on this list).  It breaks
compatibility with our older .tbz2 package format, so at this point it's
intended for testers only.  Here's an outline of all changes:

Second-generation .tbz2 format
==============================

Our new second-generation .tbz2 format features a new double-envelope design
for enhanced performance.  Package metadata is encoded in our own lightweight
and fast "xpak" format and appended to the end of the .tbz2 archive, so that it
can be quickly and easily read by our package installation programs.  Packages
can still be installed this way if you're in a pinch (although nothing will be
recorded in the package database):

# cat mypkg-1.0.tbz2 | bzip2 -dq | tar xvf - -C /

All the ebuild components have been updated to to use this second-generation 
.tbz2 format, which will be the default package format for Gentoo Linux 1.0 and
beyond. ***Portage 1.5_pre1 is incompatible with old-style .tbz2 archives***

Enhanced package metadata
=========================

Binary packages as well as packages merged from sources now record additional
helpful metadata in /var/db/pkg.  Right now, we're recording the following information:

/var/db/pkg/sys-apps/foo-1.0
	
	CFLAGS		Optimization flags used to compile foo-1.0
	CXXFLAGS	C++ optimization flags used to compile foo-1.0
	CHOST		The setting of CHOST used to compile foo-1.0
	USE		The "use" settings enabled at the time foo-1.0 was compiled
	PROVIDES	The virtual packages that foo-1.0 provides
	CATEGORY	Package category
	PF		Full package name
	RDEPEND		Runtime dependencies
	LICENSE		License information

You'll notice a new "LICENSE" setting.  Developers can begin to add a LICENSE field
to their ebuilds and use it to record the licensing information for the package. 
Examples:

#you'll use this most of the time
LICENSE="gpl"

#for BSD-like stuff
LICENSE="bsd"

#if a package offers multiple licensing options
LICENSE="gpl artistic"

Recursive pkgmerge
==================

Thanks to our second-generation .tbz2 format, "recursive pkgmerge" now becomes
easy.  Our new, enhanced pkgmerge command now performs the following steps:

# pkgmerge foo-1.0.tbz2

	1. Scan foo-1.0.tbz2 for runtime dependencies
	2. If any, scan PKGDIR, creating a directed graph of all runtime dependency information
	3. Begin installing all necessary packages in optimal order

To use pkgmerge, you'll need to set PKGDIR to point to the location of your
.tbz2 packages.  PKGDIR defaults to /usr/portage/packages.  Portage expects to
find an "All" directory inside PKGDIR that contains all the packages.  Please
remember that the new pkgmerge is incompatible with our old package format, so
you can't use this to recursively merge your current (old-style) packages.
However, Gentoo Linux 1.0_rc4 will use this new packaging format, and recursive
pkgmerges will be fully supported :)

pkgmerge also supports a new option, "--pretend".  If you specify it as the first
argument, pkgmerge will tell you the packages it would install (in order), rather
than actually installing them.

Safe auto-unmerge
=================

You probably know that if you try to merge an already-installed package, the
current (old) version of ebuild doesn't let you.  Instead, it asks you to
unmerge the currently-installed version first.  This doesn't happen anymore.
Instead, the original instance of the package will be automatically safely
unmerged after the new instance has been merged.  So, if you have gimp-1.2
already installed, but you update your gimp-1.2.ebuild with a few fixes, you
can simply type:

# ebuild gimp-1.2.ebuild merge

The new version will be compiled and merged, and then the older instance of
gimp-1.2 will be automatically and safely unmerged, removing any files no
longer used by the new instance.

ebuild updates
==============

The "ebuild" program itself has been updated in several ways.  First, the
"remerge" command has been removed (it's no longer needed, just use "merge").
Secondly, Achim's FHS 2.1 fixes have been incorporated into Portage, so that
all documentation and man pages (installed using doman and dobin only) will end
up in /usr/share rather than /usr/.

python updates
==============

We have a new module called xpak.py that handles the creation and parsing of
xpak metadata.  The portage.py code has been majorly overhauled.  We now use an
object-based "dblink" system to access the /var/db/pkg object database.  By
providing this object API, in the future we will be able to re-implement the
database code in C/C++, write a mySQL package database backend, or do whatever
we like.  Of course, our new .tbz2 format is package database independent,
unlike old ebuilds that directly reference /var/db/pkg.

There are also a host of other cleanups and improvements, resulting in a
portage.py that has a bunch of new functionality, but is actually somewhat
_smaller_ than the original version.

new file locations
==================

All Portage binaries now reside in /usr/lib/portage/bin, and symlinks are
created in /usr/sbin and /usr/bin for popular binaries.  ebuild is now in
/usr/sbin, along with pkgmerge.  The new tbz2tool and xpak binaries live in
/usr/bin.  If you're going to test Portage 1.5_pre1, be sure to delete 
your old /usr/bin/ebuild script!

That's all for now :)

Best Regards,

-- 
Daniel Robbins					<drobbins@g.o>
President/CEO					http://www.gentoo.org 
Gentoo Technologies, Inc.