[gentoo-dev] EAPI and PMS for people who haven't been paying attention

[Ciaran McCreesh]

It looks like way too many people don't understand EAPI or PMS. In the
interests of reducing the amount of noise these people make whenever
anyone mentions either, I give you the following guide.

EAPI and PMS for People Who Haven't Been Paying Attention
=========================================================

This isn't a formal specification, and is not technically accurate.
Most of this is massively oversimplified. It is focused at developers
who aren't themselves directly involved in either package manager
development or design of future EAPI features -- it's too technical for
end users, who only need two sentences, and not technical enough for
people who're deeply involved.

In basic terms
--------------

All ebuilds have an EAPI. The EAPI is a piece of metadata that tells
the package manager what rules to follow when using that ebuild.
Currently existing EAPIs are '0' and '1'. EAPI doesn't stand for
anything.

PMS is the Package Manager Specification. It will specify and document
EAPIs '0', '1' and any future EAPIs, along with tree and profile
layouts and various related requirements.

EAPIs allow package managers to add new features without breaking
systems using older package managers. They also allow package managers
to change behaviour of existing functionality that only works the way
it does by fluke, without having to support ebuilds that rely upon
implementation flukes that were never supposed to be part of the ebuild
API.

For example: say a new Portage version comes along with a new dep
resolver that's more efficient. But because it uses a different
algorithm, it behaves differently with respect to blockers. Having a
well defined specification for EAPIs provides a clear cut definition of
whether it's ok for Portage to do that (and thus whether any ebuilds
that rely upon flukes of the old behaviour must be fixed), or whether
it's a Portage bug.

Package managers that don't support an ebuild's EAPI mustn't attempt to
do anything with that ebuild. They could, for example, display the
ebuild as masked in a special way. This removes a lot of (but by no
means all of) the upgrade problems and arbitrary "we must wait X months
since a release before using new features" guidelines.

Ebuilds have EAPI '0' unless they explicitly request EAPI '1'. The EAPI
'' (empty string) is equivalent to EAPI '0'.

EAPI '0' is what you all know and love. New features won't be going
into it. Most, but not all existing ebuilds that aren't EAPI '1' will
be EAPI '0' compliant. Some existing ebuilds won't be, since
realistically they're relying upon behaviour that can neither be
expected to remain the same in future Portage versions nor be
independently reimplemented.

EAPI '1' is EAPI '0', except for the following changes:

- Slot deps are allowed.
- IUSE defaults are allowed.
- The default src_compile works with ECONF_SOURCE.

EAPI '1' was introduced to allow developers to use functionality that
was available in Portage at that time. It is defined in terms of
changes to EAPI '0'. It does not contain lots of useful things that
people want or need because Portage support was not available at the
time of writing. It is likely that an EAPI '2' will be introduced at
some point that also does not contain lots of useful things, simply
because offering some of the things that people want now is better than
offering all of them later.

You will note how EAPI '1' can easily be specified despite EAPI '0' not
being completely well defined.

Profile files, etc, don't have an EAPI. You can't use slot deps
anywhere in the tree except in ebuilds (see later for eclasses).

EAPI doesn't specify package manager configuration, VDB or client
behaviour. Ebuilds mustn't rely upon any of these.

Portage doesn't, in general, do validation. In particular, it won't
moan if you use EAPI '1' features in an EAPI '0' c/p-v. You have to be
careful with this.

EAPI in more detail
-------------------

EAPI values are strings. They may or may not be integers. They cannot
be compared for anything except equality. The next EAPI might be called
"larry-on-wheels".

EAPI is metadata for a C/P-V. It is constant across the entire C/P-V,
and is subject to the usual metadata invariancy and extraction rules.
Currently EAPI is set by the EAPI variable.

This is an issue with eclasses. It means that:

* If EAPI is to be changed, it must be done so before any inherit.

* eclasses must not modify EAPI.

* eclasses must work with all EAPIs, and may not require specific EAPIs
themselves. They may test for EAPI and do different things depending
upon EAPI, so long as those things do not involve erroring out at any
stage. So you can do this in an eclass:

    if [[ "$EAPI" == "1" ]] ; then
        DEPEND="foo/bar:2"
    else
        DEPEND="=foo/bar-2*"
    fi

  But not this:

    EAPI="1"
    DEPEND="foo/bar:2"

  Or this:

    if [[ "$EAPI" != "1" ]] ; then
        die "EAPI must be 1"
    fi

  Or this:

    foo_pkg_setup() {
        [[ "$EAPI" == "1" ]] || die
    }

Yes, this is not ideal. No, you can't have it changed for EAPI 0 or 1
no matter how much you moan about it (and if you don't understand why
not, you really have no business discussing it).

This is a bigger issue with global scope commands. It means that future
EAPIs can't change behaviour of any global scope callable command, nor
can they introduce new global scope callable commands, nor can they
change how EAPI is exported, nor can they add new behaviours or
environment options that are available for global scope. This means
that you can't even have the above modified for EAPI 2. Fixing this
requires the suffixes GLEP (and if you don't understand why, kindly
avoid wasting everyone's time talking about it).

PMS in more detail
------------------

PMS is a PDF document that will cover EAPIs 0, 1 and any future EAPIs
that come along. At some point in the future, following much extension,
proof reading and verification, a particular build of PMS will be
submitted to the Council, who may or may not approve it. When new EAPIs
come along, and when flaws are discovered in PMS, an updated version
will be submitted to the Council, who again may or may not approve it.

This means that:

* The stuff in any particular scm repo is not what the Council will be
asked to approve.

* The definition for EAPI '0' will probably change a bunch of times.
This won't be to add new features or to change behaviour. Rather, it'll
be because we'll find problems with the base specification. In
particular, without independent implementations, it's extremely hard to
catch many of the things upon which ebuilds rely.

Case in point: econf has to be a shell function, and emake has to be a
standalone binary. We wouldn't have caught that by working off just one
package manager.

More obscure case in point: $A mustn't contain trailing whitespace. We
wouldn't have caught that one either...

So far as I'm aware, everyone who's currently working on PMS is working
off this repository:

    http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git

(Sidenote: This may change soon -- when we moved to Gentoo hosting, we
were assured that Gentoo infra had cleaned up its act and would no
longer be abusing root access for political reasons. Recent events
might make us have to reconsider this decision...)

Patches can be submitted to pms-bugs@g.o. It's probably wise to
talk to me before starting work on something, in case you're about to
modify something that someone else is working on. Case in point: I'm
currently redoing the env stuff into a nice big table that clearly
shows phase validity, value preservation and the like, so if you submit
any patches that collide with that I'll Linus you, and we don't have an
Andrew Morton.

-- 
Ciaran McCreesh