Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

On Fri, Sep 02, 2005 at 10:53:05AM +0200, Paul de Vrieze wrote:
> On Friday 02 September 2005 08:04, Brian Harring wrote:
> >
> > Like I've said, EAPI is ebuild specific.  Ebuild is a format; eapi
> > defines revisions of it, in my mind a minor revision of the ebuild 1
> > format.  Any form of loss of backwards compatability *should* be a
> > different format, .ebuild2 for all I care.
>
> The new proposed format loses backwards compatibility. If there is
> backwards compatibility no new format or api version is needed. EAPI
> should work on the python level, not only on the ebuild.sh level.

The two are utterly intertwined.

Doesn't matter if portage thinks it's eapi 1 or 2, what matters is if the 
ebuild*sh env *creates* an eapi 1 or 2 env for execution.  Python side 
just behaves a bit differently, ebuild*sh is where all of the actual 
execution occurs (and varies dependant on eapi).

> > Trying to use EAPI to allow for N different formats into one format is
> > wrong from where I sit; you would need a container format for it,
> > which ebuild wasn't designed for (nor is it easily extensible to be
> > made so I posit).
>
> No it would state that the eclass is 100% compatible with both by the
> formats overlapping and the ebuild not threading outside the overlapped
> area.

And how is this going to work when it's more then just a split of a 
func?  

I'm not saying it can't be done, I'm saying it *shouldn't* be done.  
It's begging for screwups; relying on portage to pick and choose 
between eapi levels for an ebuild's execution env dependant on an 
eclass *requires* the ebuild to be fully compatible to those 
eapis, same for the eclass author.  That's not so easy to do, and will 
lead to a buttload of case tests of EAPI, which isn't going to improve 
the code (imo).

People aren't going to do it, is my opinion.  It's unnecessarily 
complex with minimal gain; the gain being decreased due to the 
inherent complexity of 
A) pulling this off in portage
B) comprehending *exactly* what portage will do in each case
C) the ebuild/eclass author not only managing to get usage of *one* 
   eapi correct, getting potentially N eapi's correct.
D) eclasses that inherit eclasses requiring the same eapi 
   compatability checks
E) within two bodies of code, devs  keeping seperated in their mind the 
   differences between each eapi, and *never* blurring them.

Note I'm not pulling the "everyone else is ignorant" crap that popped 
up on -dev ml; I'm stating that writing to an api is manual work, and 
people will screw up on it, just the same as in writing the
implementation of that api, *we* will screw up on it.

Bugs per line, it's unavoidable.  Not advocating that we dumb 
everything down, advocating we don't add something that (assuming we 
get it right in implementation) is one hell of a pandora's box from 
the standpoint of complex ebuilds combined with complex eclasses.

That's also not even getting into the fact that what you're proposing, 
effectively greping EAPI from the file is 

A) helluva lot slower on regen
B) trying to turn ebuilds into a container format.  They're not a 
   container format, nor should they be (kitchen sink shouldn't be 
   included).
C) core's more then capable of supporting seperated formats; the 
   'container' should be the repository, not jammed within the package 
   data.
D) It's a helluva lot easier, and less chance for screwup to just 
   define a new format, use a new extension, and have the repository 
   implementation use a different pkg format for that file.
E) New, non eapi=1 style change with a different extension wouldn't 
   even be *seen* by the noncompliant repositories.  Helluva lot 
   simpler.

Not advocating we define a new format every day; I'm advocating that 
we define a format (v1), we do tweaks to it, and define a new 
*seperate* format when we need stuff that is beyond the keen of the 
existing format.  Shifting away from declarative syntax, moving away 
from bash syntax, etc.  If it isn't akin to our existing 
definition of an ebuild, it's not an ebuild and should *not* be jammed 
into the ebuild format.

It's a new beast, and should be seperated.

That's also ignoring the additional interaction of elibs thrown into 
the mix.  If what you're suggesting is implemented, eclasses *and* 
ebuilds would be litered with 

if [ "$EAPI" == 2 ]; then
	eapi2 specific stuff
elif [ "$EAPI" == 1]; then
	eapi1 specific stuff
else
	general, all eapi stuff
fi

Note that the code above also has a nice hidden bug in it that's not 
obvious till you think about it; there is no way to predict at the 
time of the ebuilds writing that EAPI3 will work for the else block, 
yet logic structures of that sort *will* exist if you open up the N 
EAPI compatibility for an ebuild/eclass.  Any bumping of the potential 
eapi's for an ebuild/eclass will expose it.

That and I'd hope if it were ever implemented, devs would be using 
functions within the logic block; even with, any non-trivial bit of 
code is going to get nasty as it's littered with a hundred and one 
checks of eapi tweaking the flow ever so slightly.

It's needlessly complex, and a recipe for disaster.


> Take for an example many simple kde applications. Those use the kde
> eclass to do all the work and only contain a skeleton of variables
> themselves. These ebuilds are compatible with both the current as the
> proposed new API. When marked so, they could be used as EAPI=1 as soon as
> the kde eclass is ported to EAPI=1. Similarly many eclasses do not
> provide src_compile, and as such are compatible with both EAPI versions.

Lacking src_compile, they're compatible.  They're also quite likely an 
elib, which is a different beast designed to address the fact that 
library code of eclasses (true library), should exist outside of the 
eclass.  Eclasses are purely templates for ebuilds, a derived 
template/class from the base template, that an ebuild uses to avoid 
defining common chunks of code.

Common code blocks, say what eutils/toolchain does are elibs (yes I 
know about the patch dep on eutils, ignore it for the sake of 
arguement).

> > EAPI's original specification was for handling addition of new funcs,
> > different hooks in the ebuild; I prefer it remain as this.  The core
> > rewrite is format agnostic, if a new format is defined (whether a
> > massively managled version of ebuild or flat out new), it's a seperate
> > format and should be handled via the core, not via ebuild specific
> > package handling.
>
> EAPI now is going to be used for the above. It can however with very
> little effort be made such that future ebuild format revisions are
> possible. Also don't be mistaken that splitting out configure and make
> stages do need support from the python part.

Well aware of the complexities involved in keeping the python and bash 
side in sync; it's part of the reason I oppose what you're proposing.

Change in terminology here btw; eapi (imo) has never been defined as a 
format revision, it's a change in the portage exported bash api, and 
how it potentially calls into the ebuild.sh api (addition of 
src_configure).

People split off a new format (seperate extension fex) when they no 
longer can make changes to the api, where grand changes are required.  
EAPI isn't designed to address that; it's designed to address 
ebuilds/eclasses requiring new ebuild*sh functionality, addition of a 
new do* or change to # of phases called.

eapi0 -> eapi1 is probably going to be the nastiest change over from 
where I sit; it's the transition from a totally unversioned ebuild*sh 
env to one that has saner handling.

Total change over of ebuild format, as you're hinting at is not eapi's 
intention; that's covered by the core being format agnostic, and 
repository abstractions.


> > There's no reason a repository can't hold multiple formats internally;
> > the capability is there, use that rather then trying to jam too much
> > into EAPI, imo at least.
>
> How would you suggest to do this then. The ebuilds/eclasses are completely
> the same except for their EAPI definition. They also have the same
> (file)name. And that is not counting the fact that two files containing
> the same is bad design as there will allways be an issue of one file
> being updated and the other not.

I'm saying we stick with what already occurs in the tree; eclass 
functionality is either prototyped outside of the tree, and 
introduced, or an ebuild has the eclass functionality prototyped 
within it, and that code migrated to the eclass.

For example- I've got kde eclass that's eapi0, and want to migrate to 
eapi1.  I break the kde eclass up so it's eapi1 compatible (at the 
func level), but with the exported src_compile eapi0 so I don't break 
the existance ebuilds.

I then add a shim eclass that *is* eapi1 (calling the chunked kde 
internals), and migrate pkgs gradually over to it.  Once it's fully in 
use, convert the underlying kde eclass to eapi1 (move the shim code 
in), leave the shim as indirection, and remove the shim from the 
testing ebuilds.

Or.

I want to migrate my kde eclass to eapi1.  Like any eclass change, I 
find what would be affected.  I break the kde eclass into eapi1 
compatible chunks (seperated configure and compile), with a eapi0 
src_compile that calls both.

I then go and clean up the ebuilds that override src_compile, and 
convert them over to the new funcs in the eclass (bumping the ebuild's 
eapi in the process).  Once they're addressed, all that remains is the 
ebuilds that use the eclasses exported src_compile; remove the eapi0 
src_compile shim in the eclass, and set EAPI=1 within the eclass.  All 
ebuilds that rely on the eclass to define eapi and the majority of 
their code (those that didn't required modification), are now eapi1, 
as is the eclass.

Heading off the "make the eclass eapi 0 and 1 compliant with portage 
choosing between them", what I'm stating is that the eclass and ebuilds 
authors do it, they be explicit and manage the conversion themselves 
(if they see the need to bump to eapiN).
They can manage the code and keep it sane, rather then trying to ifdef 
N different eapis and hoping portage doesn't screw up (it will screw 
up invariably, as will they in both my solution and yours).  

Mine just is simpler, from where I sit, less chance of screwups beyond 
usual bonehead mistakes people always make.

Either solution expects ebuild/eclass devs to get it right; having 
portage somehow swivel the eclass so that it's exported api/env is of 
a certain eapi expects the devs to have that code working properly, 
and for a *perfect* implementation that can do this without screwup.

It's not realistic, frankly.
~harring