[gentoo-dev] Re: Re: Re: Re: Re: [GLEP] Use EAPI-suffixed ebuilds (.ebuild-EAPI)

[Steve Long]

Ciaran McCreesh wrote:
> On Thu, 20 Dec 2007 00:07:35 +0000
> Steve Long <slong@××××××××××××××××××.uk> wrote:
>> Do you think a generated EAPI is a good idea? I'm curious as to
>> how that would be reflected in the filename (as well as your reasons
>> ofc.)
>
> I'm suggesting that if EAPI is a variable, there are use cases for being
> able to set the variable in ways other than right at the top of the
> ebuild. If it isn't a variable then those use cases aren't relevant.
>
Point is that your filename format restricts it in exactly the same manner.
So let's just stick with the use cases which /that/ supports, which can
more easily be supported with the original design and the simple
restriction people keep asking for.

>> No: without knowing the EAPI when generating said data. If that needs
>> to be known relatively soon in order to generate the rest, extract it
>> early. You still only need to load the file from disk once, and you
>> don't need to source to get that data, given one simple restriction
>> (only declaring it once.)
>
> You can't get the EAPI from the ebuild without knowing what EAPI the
> ebuild uses. That's one of the points you're missing.
>
Wow that doesn't half sound like nonsense. On the one hand you want a
restricted filename suffix, on the other it *has* to be full bash parsing.

An EAPI="blah" at top-level in the file is exactly the same as the suffix in
terms of what can be represented (actually more capable: it also enables eg
prefix EAPIs which I understand was one of the main motivations to extend
the format away from a defined ordering.) If the EAPI introduces a further
eapi function, all well and good; the same datum is still required, whether
kept in the filename or in the ebuild.

According to the original discussion this was always supposed to be a
variable set in the ebuild source:
"We would like to introduce a new ebuild variable named EBUILD_FORMAT,
that tags the ebuild with a specific ebuild API version it provides or
uses."[1]

At that time others made the case for restricting its format in exactly the
same way you have been resisting for the ebuild source, but see as fine for
tacking onto the end of the filename:
"I would also suggest requiring that EAPI can be retrieved by a simple 
line by line parsing without using bash. (This allows for changing the 
parsing system)"[2]

The idea of a different suffix was discussed back then as well:
"portage *should not* ignore those ebuilds.  If the user 
wants to merge something that is a later ebuild api then they have, at 
least portage chucks an exception that the UI can wrap into 'upgrade 
portage'.

With what you're proposing, we instead get bugs about portage missing 
packages."[3]

(That was written when there were no other PMs besides portage3/pkgcore)

http://www.mail-archive.com/gentoo-dev%40lists.gentoo.org/msg03946.html
has a fuller discussion.

> It's an ebuild issue, not a package manager issue.
>
You keep saying that; sure EAPI is visible to ebuild and maintainer (doh)
but the reason you have stated for this change in file naming (which is a
lot more far-reaching than a simple restriction on the format of a
variable) is so that the *PM* doesn't have to source the ebuild to get the
EAPI.

Several people have independently suggested the same solution that would
work fine in the existing framework, which was in fact proposed in 2005:
"Mainly, limiting the syntax has the undesired affect of deviating from 
what users/devs know already; mistakes *will* occur.  QA tools can be 
written, but people are fallable; both in writing a QA tool, and 
abiding by the syntax subset allowed."[3]
I don't think those are really an issue nowadays; devs know to run repoman,
and so do sunrise submitters.

I haven't seen one lucid explanation from you for why it wouldn't work,
beyond ramblings about requiring full bash sourcing capability for what you
yourself envisage as being a static variable, fixed per file.

>> I accept it would make metadata generation quicker, but the
>> end-user can already use -metadata-transfer atm and never need to run
>> that process. It would save many more cycles if more users did imo
>
> You're confusing the two different types of metadata. Which again shows
> that you need to start understanding the metadata process before trying
> to discuss design decisions relating to it...
>
It doesn't make any practical difference[4]: the computational issue is how
to avoid a source statement via bash from another language.

I note in passing that a metadata cache is available, with no runtime
requirement on the user's part, since it is taken from the rsync'ed data.

>> (optimising early here seems silly tbh, given that paludis now
>> requires ruby.)
>
> Eh? Now what're you on about?
>
http://bugs.gentoo.org/show_bug.cgi?id=198864

>> Given that, as a user I see no benefit to my machines that would
>> justify the maintenance headache which I know as a coder this will
>> result in.
>
> There is no maintenance headache.
>
Yeah, keep saying it. That'll make it true..

>> Quite apart from all the changes to supporting tools and workflow,
>> it's pushing an implementation-specific datum into a namespace where
>> it's neither needed nor useful, apart from to the implementation.
>
> It's an ebuild issue, not a package manager issue.
>
Hmm see above. I note you accept that it requires changes to supporting
tools and workflow.
 
>> >> > Ebuilds are not config files.
>> >> >
>> >> Indeed not, but they can be parsed as such for simple, core,
>> >> metadata.
>> >
>> > There is currently no information available from an ebuild that can
>> > be parsed by any tool other than bash.
>> >
>> OK, but restricting EAPI= across the board (in the way that others
>> have already asked for) and enforcing it via repoman would mean EAPI
>> could easily be extracted.
>
> Except that it's an arbitrary, pointless restriction.
>
Er arbitrary, no, since in practise it means exactly the same thing as the
filename suffix (one single string declaration.) Pointless not at all,
since it allows you to avoid calling bash and waiting for it to source,
without an ugly hack. It also keeps the existing work practises that
everyone is used to and doesn't mandate any changes to support tools.

Given that what we currently have actually supports more than the suffix
does, I would class the proposal as pointless restriction, requiring
code-changes for zero benefit to devs and users, while arbitrarily setting
the prefix project back. And let's not pretend that making code changes
across the board will be that easy; regressions are no fun, and nor are new
bugs, especially when they result from changes imposed for no technical
merit.

>> Hmm I think you should consider the example that this sub-thread is
>> about, and whether an apology would be in order.
>
> Huh?
>
Gee is it really that hard to look back at the example from your colleague
that started this sub-thread? Yet you expect everyone else to keep track of
every delightful comment from you.. *sigh*
 
>> Since only declaring it the once /seems/ ok with you, what on Earth
>> is so hard about extracting it?
>
> With the current situation, the EAPI has to be known for the EAPI to be
> calculated. Adding in a pre-pass layer enforcing new file format
> restrictions does not solve the problem we're trying to address.
>
There is no pre-pass layer enforcing restrictions (nice FUD though); repoman
or QA-toolFoo would do the check before the ebuild even got into the tree.
And again, as others pointed out when this was first discussed, and still
others have pointed out on this list, it really isn't that hard to get a
simple EAPI="foo" out of a file, and the restriction is exactly the same
for the maintainer as with your hacked on suffix. The QA check before
committing can trivially enforce the singleton restriction and we can use
EAPI as it was intended with no change to workflow and much less work.


[1] http://www.mail-archive.com/gentoo-dev%40lists.gentoo.org/msg02668.html
[2] http://www.mail-archive.com/gentoo-dev%40lists.gentoo.org/msg03868.html
[3] http://www.mail-archive.com/gentoo-dev%40lists.gentoo.org/msg03873.html
[4] But thanks for the explanation, that really cleared things up. I'll
assume you're talking about metadata generation during depgraph resol'n
then, until you scold me for misunderstanding again. And I note that you
have been disparaging of discussion of bash optimisations in eclass/ebuild
code, yet are falling over yourself to give everyone else a load of work to
speed up what I thought was already the fastest PM known to humanity.


-- 
gentoo-dev@g.o mailing list