Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ralph Sennhauser]

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller <ulm@g.o> wrote:

> Hi all,
>
> The way how we currently specify the EAPI in ebuilds has some
> problems. For example, there is no sane way to allow usage of features
> of a new bash version in a new EAPI. So we are currently stuck with
> bash 3.2. Also changes of global scope behaviour, like addition of new
> global scope functions (similar to "inherit") are not possible.
>
> These flaws are outlined in GLEP 55 [1]:
> | In order to get the EAPI the package manager needs to source the
> | ebuild, which itself needs the EAPI in the first place. Otherwise it
> | imposes a serious limitation, namely every ebuild, using any of the
> | future EAPIs, will have to be source'able by old package managers
> | [...]
>
> The council has voted down GLEP 55 more than a year ago, but at the
> same time requested that a solution for the mentioned issues should be
> found. [2] However, there was no progress since then.
>
> The issue arose again in bug 402167 [3] where several solutions have
> been discussed. Below, I try to summarise the possible options
> resulting from that discussion.
>
>
> *** Proposal 1: "Parse the EAPI assignment statement" ***
>
> This first proposal would require that the syntax of the EAPI
> assignment statement in ebuilds matches a well defined regular
> expression. A scan of the Portage tree shows that the statement only
> occurs in the following variations (using EAPI 4 as example):
>
>    EAPI=4
>    EAPI="4"
>    EAPI='4'
>
> Sometimes this is followed by whitespace or a comment (starting with
> a # sign). Also, with very few exceptions the EAPI assignment occurs
> within the first few lines of the ebuild. For the vast majority of
> ebuilds it is in line 5.
>
> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
>   have been suggested).
> - The statement must match the following regular expression (extended
>   regexp syntax):
>   ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
>
> Note: The first and the third point are already fulfilled by all
> ebuilds in the Portage tree. The second point will require very few
> ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
>
> The package manager would determine the EAPI by parsing the assignment
> with above regular expression. A sanity check would be added. Citing
> Zac Medico in [3]: "The fact that we can compare the probed EAPI to
> the actual EAPI variable after the ebuild is sourced seems like a
> perfect sanity check. We could easily detect inconsistencies and flag
> such ebuilds as invalid, providing a reliable feedback mechanism to
> ebuild developers."
>
> This proposal comes in two variants:
> 1a) The change is applied retroactively for all EAPIs.
> 1b) It is only applied for EAPI 5 and later (which means that the
>     result of the EAPI parsing would be discarded for earlier EAPIs).
>
>
> *** Proposal 2: "EAPI in header comment" ***
>
> A different approach would be to specify the EAPI in a specially
> formatted comment in the ebuild's header. No syntax has been suggested
> yet, but I believe that the following would work as a specification:
> - The EAPI must be declared in a special comment in the first line of
>   the ebuild's header, as follows:
> - The first line of the ebuild must contain the word "ebuild",
>   followed by whitespace, followed by the EAPI, followed by
>   end-of-line or whitespace.
>
> Again, the proposal comes in two variants:
> 2a) It is combined with a one time change of the file extension, like
>     .ebuild -> .eb.
> 2b) The usual EAPI assignment statement in the ebuild is still
>     required, at least for a transition period.
>
> In the 2a case, the EAPI variable could be made read-only in bash
> before sourcing the ebuild. In the 2b case, a sanity check similar to
> the one mentioned above would be added.
>
>
> What do you think?
>
> (I really hope for a constructive discussion here. So, if you want
> to comment that all of the above proposals suck and GLEP 55 is much
> superior, then please open a new thread for it.)
>
> Ulrich
>

Currently 5 proposals are listed on the wiki. [4]

While all of them have some temptations the actual goal is to make
obtaining the EAPI the very first step so everything else can be
defined in terms of EAPI and so immediately deployable in future. This
are changes in atom syntax like needed for GLEP 54 or those bash
feature often mentioned besides many other things one can think of. 

GLEP 55 requires changing ebuild extensions on a regular basis but
doesn't impose any limit on the ebuild format or atom syntax, only the
file extensions would be imposed. The ebuild extensions for GLEP 55
would likely always be ebuild-<integer> as integers are reserved for
future use by Gentoo. While for example .ebuild-5 is still recognised
as an ebuild; .eb .ebld .ebd .bld .dliube .dlbe .be are not. This
brings me to the point if not GLEP 55 then only if we can keep .ebuild.
That one-time extension change thingy smells like being stubborn.

If we aren't going to to change the extension the most sensible thing
is to pack the eapi into the ebuild as done now but get rid of as many
limitations as possible. Looking at the proposals for eapi specification
in ebuilds the header comment is the clear winner here. The eapi can
also instruct the package manager to remove the first line for further
use of the ebuild if required for obscure formats / languages.

I already ruled out proposal "3 EAPI in header comment and one-time
change of file extension" for me as an option, see above. And "2 EAPI
in header comment" comes with compatibility issues for an undefined
amount of time.
If we want to keep .ebuild but avoid the compat issue another variant
would be "EAPI in header comment and one-time change of ebuild
location" or more formal:

6 EAPI in header comment and one-time change of ebuild location:

- add a directory $CATEGORY/$PN/ebuilds to ebuild repositories.
- all files in $CATEGORY/$PN/ebuilds are ebuilds and are using a well
  defined first line to denote the EAPI.
- For practical reasons the header should be a bash comment. PMs
  shouldn't have to remove or skip first line from file for further
  processing of ebuilds supporting bash comments.
- the .ebuild extension can be kept but could be changed if ever
  desired. This due to the filename only having meaning if the
  EAPI of the file is known.

Comparing this with GLEP 55 then this allows us to keep .ebuild in
return of some overhead with roughly the same pros and cons otherwise,
right?

Ralph


>
> [1] <http://www.gentoo.org/proj/en/glep/glep-0055.html#problem>
> [2]
> <http://www.gentoo.org/proj/en/council/meeting-logs/20100823-summary.txt>
> [3] <https://bugs.gentoo.org/show_bug.cgi?id=402167>
>
[4] <http://wiki.gentoo.org/wiki/Alternate_EAPI_mechanisms>