Re: [gentoo-pms] Clarify wording on self-blockers

[Maciej Mrozowski]

On Monday 02 of May 2011 03:41:43 Brian Harring wrote:
> On Wed, Apr 27, 2011 at 11:34:00PM +0200, Maciej Mrozowski wrote:
> > On Wednesday 27 of April 2011 21:24:51 Ciaran McCreesh wrote:
> > > > > Uh, no, that's the entire point of EAPIs.
> > > >
> > > > This is a common misconception.
> > >
> > > Uh, no. You are completely and horribly wrong.
> > >
> > > EAPIs were introduced to avoid the problems that we used to have where
> > > different versions of Portage did different things with the same input
> > > (including, but not limited to, difficulties in adding new features).
>
> To be clear, Ciaran is very much correct here- I added EAPI in to
> fix ongoing shit like this where overlays/old ebuilds in the
> tree would be broken via the latest/greatest insanity portage leveled,
> and to allow new ebuild functionality to roll out quickly instead of
> having to sit for at least a year.
>
> This is *exactly* the point of EAPI- it's format versioning for
> compatibility.  As such once a version is considered "released" or
> "stable", it's basically immutable.
>
> EAPI was explicitly designed to allow a portage from 2 years back to
> be able to function w/ ebuilds written to that EAPI- assuming the
> version of portage in use was compliant w/ the EAPI2 (from 2 years
> back), it should still function w/ ebuilds written in current times
> that are EAPI2.

And wrt blocks it would still work. It's not like clarifying self-mutual 
blocks semantics would magically alter behaviour with regards to how the 
package is merged. It may only have consequences on package set level, not 
ebuild level. But then again, IIRC Paludis still doesn't perform auto-
unmerging of not referenced soft-blocked packages and it's being extensively 
used in gentoo-x86 tree and I see nobody screaming...
(this may have changed however, my point is invalid if that's the case) 

> Not saying it was done *perfectly*, but this is a large part of it's
> intent.
>
> > > > From ebuild developer point of view - EAPI specification should
> > > > provide all knowledge required to create ebuilds along with detailed
> > > > and unambiguous description of what to expect from package manager
> > > > that is compliant with said EAPI.
> > >
> > > Oh heck no. That really isn't the point at all.
> >
> > But it is. Discussion about blocks is precisely about what is expected
> > package manager behaviour for self-mutual blocks (in EAPI>=2) in such
> > case. And what you propose is 'behaviour is undefined for self-mutual
> > blocks' - if so, why are they permitted by specification and not banned
> > instead? The way you understand the word 'specification' doesn't really
> > allow any specs bugfixes since they would "magically" invalidate any
> > existing implementations, unless there is just one (and actually there
> > *is* just one *supported*) so that specification could be kept in sync
> > with it.
>
> Ciaran's coming at this from the academic definition of
> "specification".  In a perfect world, it's correct- once a spec is
> stabled, it's immutable sans clarifying wording/intent, and even that
> involves occasionally converting bad parts of the spec to unspecified
> (or unspecified if a large PM screwed up their implementation badly
> enough you can't rely on that assertion any longer).
>
> Pragmatism is such that bugfixes/tweaks do get pushed in where
> required, and where it doesn't fuck us hard to do so.
>
> As for *why* they're allowed in the first place, it's because the spec
> is incomplete- it has a large number of assertions, but covering all
> of it is fairly hard (and questionable in usefulness for certain
> spots)- I generally hate 'unspecified', but the primary usefullness is
> to make clear that proceeding behaviour could vary across PMs-
> specifically it should be used when
>
> > I thought PMS is specification and not a documentation - hence it should
> > specify (to document would be poor choice of words.. ) intended, widely
> > agreed behaviour and *defined* behaviour so that ebuild developer knows
> > how to write ebuilds that work against PM *compliant* with specification
> > and package manager developer knows how to write *compliant* PM. And
> > because there are many places where PMS is too vague, it misses those
> > goals.
>
> Asserting all behaviour is actually fairly hard.  Patches welcome for
> that intent, but additions to the spec have to be written to avoid
> breaking previous assertions in the spec- as said, a portage that was
> EAPI2 compliant 2 years back needs to work with ebuilds that EAPI2
> from today.
>
> It sucks, but this is /proper/ versioning.  It's a hard problem.
>
> > > > Old package managers and their expected behaviour is irrelevant as
> > > > soon as migration path to recent version exists for them.
> > >
> > > Again, you are deeply confused. It is required that an upgrade path for
> > > old package managers is kept around for as long as possible by not
> > > using newer EAPIs for certain key system packages. This is entirely
> > > different to what you're saying -- it means that certain packages
> > > should be kept at low EAPIs for as long as possible, not that EAPIs are
> > > whatever the latest Portage version does.
> >
> > No, you're making it up here.
>
> This is actually required.  Abided by?  Not as well as it should
> (python has been fucking that one up hard), but take a look at glibc's
> eapi, gcc, binutils... etc.
>
> This is *exactly* what we intended when EAPI was added 5.5 years ago
> (and was the intent for the year or so before a patch got pushed in).
>
> > > > Since intended behaviour for normal vs strong blocks is like Ulrich
> > > > specified, and migration path to the most recent from first portage
> > > > supporting EAPI-2 exists - argument to block specification update is
> > > > invalid.
> > >
> > > That doesn't follow at all. What happens if someone has an old Portage
> > > installed and the migration path includes things that rely upon a
> > > changed behaviour?
> >
> > It wouldn't be a migration path, would it?
>
> You're proving his point.

Not necessarily.
Migration path to the version of <insert your favourite package manager here> 
that is compliant with updated specification doesn't have to involve relying 
on changed behaviour (changed at this point only in specification since old 
package manager is being used to update to the newer one after all).

And why do I argue on effectively dropping support for older versions and to 
keep specification in sync with the latest version as much as possible (across 
package managers)?

Maintenance cost.
Every new EAPI requires added codepaths and it adds potential complexity if 
for instance dependency resolver behaviour is altered. If every little 
specification cleanup was conducted by bumping EAPI, it would lead to 
unmaintainable code (I know people like spaghetti though).
Well, just look at portage - only few are brave enough to touch it and even 
fewer know how it really works.
And since Gentoo lives in $0 budget working environment, resource constrains 
(time * manpower) should be considered. Since there's no budget for rewrite, 
we should simplify instead.

> > > Since you're trying to retroactively define a
> > > particular behaviour for all EAPIs that doesn't match what some old
> > > Portage versions do, your upgrade path is screwed.
> >
> > No, I'm supporting *defining* behaviour as it's intended so that package
> > managers are able to know whether they are correct. You're making all of
> > them correct by explicitly allowing them to do anything here and there -
> > it's not helping.
>
> What you're not getting is proper format support.  Yes, there are
> areas in eapi0, 1, 2, etc, that I'd *love* to go back and make a
> clearer definition on- the problem is ensuring that all versions of
> portage/pkgcore/paludis that supported those EAPI's are still
> compliant.  If that can be assured, then, within limits, we can
> redefine previous EAPIs- because effectively we're not actually
> defining any *new* behaviour, just clarifying old behaviour that
> happily matches what we want now.
>
> If those things hold, we can't do it, and that's a fact of defining
> versioned formats.
>
> > Ulrich: maybe we should consider renaming PMS to PMD? :P
>
> It certainly would be nice being able to avoid stating "active in
> gentoo's PMS process" for folks resumes, I must say (which is where a
> lot of the juvenile humour derives from I suspect).

Missed. I never meant to state I was active in Gentoo PMS process, "we" comes 
from wearing certain hat if it wasn't clear enough (that being said I just 
prefer not to use @gentoo.org in mailing lists just to observe how people 
react to "third-parties").

> So... rather than continuing to argue about what EAPI was intended
> for, lets switch back to whether or not we can clarify the rules for
> self blockers in existing EAPIs, and what we'd like it to be for
> >=EAPI5..

Yes.
The problem is that person most interested in participating in such 
discussion, the one solely responsible for reference and supported package 
manager implementation (for Gentoo) is Zac, but he's not interested in (or 
doesn't have time for) bickering with Ciaran (who apparently does have time) 
and I don't blame him at all frankly (I don't blame Ciaran either, he could 
just try to avoid stating his opinion on how broken Portage is on every single 
occasion he has like anyone even cared.. and send patches instead).

Anyway Ciaran diverged discussion to what's the purpose of EAPI and what's 
not. Not sure what's the time line for "in the old days Portage treated all
blockers as being a bit like what strong blockers are now" is, to my knowledge 
in all Portage versions supporting EAPI-2 this behaviour is consistent, well-
defined and as such could be documented as in effect for >=EAPI-2 (with 
possible altering in EAPI-5 - for instance non-strong self-blocks forbidden or 
sth)

-- 
regards
MM