Re: [gentoo-dev] Gentoo Council Reminder for May 28

[Tiziano Müller]

Am Mittwoch, den 27.05.2009, 22:43 +0100 schrieb Roy Bamford:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 2009.05.27 21:06, Ciaran McCreesh wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On Wed, 27 May 2009 20:55:33 +0100
> > Roy Bamford <neddyseagoon@g.o> wrote:
> > > That means the EAPI needs to be extracted before the ebuild is
> > > sourced, which from the figures bandied about on the list may take
> > > marginaly longer but its a price worth paying for a sound system
> > > design. Gentoo should not repeat the VHS vs Betamax war. For those
> > > who do not remember, VHS was the better marketed but inferior
> > > technical solution that won the standards war for domestic Video
> > > recorders.
> > >
> > > The aims of GLEP 55 are good but the proposed implementaion is bad
> > > practice, so GLEP 55 should be rejected in its present form.
> >
> > You have not made a single technical argument in your entire message.
> > Please don't add yet more noise to the discussion.
> >
> > - --
> > Ciaran McCreesh
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.9 (GNU/Linux)
> >
> > iEYEARECAAYFAkodnVkACgkQ96zL6DUtXhGerACcC2khWKdGKCaMTi7zE/jYyUUw
> > bU8AnA5Gg6bDz/JiDIwMB98R5t9dQNLg
> > =bfse
> > -----END PGP SIGNATURE-----
> >
> Ciaran,
>
> You chose to ignore "Adding metadata to the filename is not required
> and is bad system design practice."
>
> I assume you agree with that as you chose to snip it, not to refute it
> with a technical argument?
>
>
>
> GLEP 55 is a poor piece of technical writing. The title
> <quote>
> Title:	Use EAPI-suffixed ebuilds (.ebuild-EAPI)
> </quote>
> should be an area to be impvoved not a possible solution that has not
> even been discussed (in the document)  yet.
>
> The abstract tells readers more about a proposed solution.
> <quote>
> This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds
> (for example, foo-1.2.3.ebuild-1).
> </quote>
> and readers still don't know the problem that it tries to address.
>
> The statement of the problem is not entirely correct either ...
> <quote>The current way of specifying the EAPI in ebuilds is flawed.
> In order to get the EAPI the package manager needs to source the
> ebuild,
> </quote>
> Nope, in order to get the EAPI, the PM needs to parse the ebuild, it
> need not source it. Thats a statement of the current design.
>
> <quote>
> which itself needs the EAPI in the first place.
> </quote>
> Well, thats what current designs do, its a design than can be changed.
>
> <quote>Otherwise it imposes a serious limitation, namely every ebuild,
> using any of the future EAPIs, will have to be source'able by old
> package managers
> </quote>
> That is true, unless the .ebuild extension is changed so we get a clean
> break with the past. It does not mean the EAPI needs to be a part of
> the file name.
>
> The GLEP discusses this and and dismisses it for no sound technical
> reasons.
>
> Until we get to the Abstract solution, the GLEP reads like a sales
> brouchre, which is what reminded me of VHS vs Betamax.
> <quote>
> A solution to this problem has to lift those limitations and the only
> way to do it is to make the EAPI of an ebuild available to the package
> managers in a way that doesn't require them to source the ebuild.
> Another important requirement is for the solution to be backward
> compatible, which has the pleasant side-effect of making the solution
> applicable in the Gentoo tree right away. A solution to this problem
> has to lift those limitations and the only
> way to do it is to make the EAPI of an ebuild available to the package
> managers in a way that doesn't require them to source the ebuild.
> </quote>
> Thats all true or highly desireable.
>
> The
> <quote>
> Hurts performance: yes
> </quote>
> needs to be quantifed and included in the GLEP, so that readers can
> make an impartial objective assessment of the alternatives on offer and
> hopefully support the best technical solution. That need not be the
> fastest.

I did some analysis on that. The result is that the the performance
penalty of not having the EAPI not in the filename depends on various
factors. But it is for sure that there is a performance penalty.

And here is why (I'm only looking at the non-degenerated case with valid
metadata, ignoring overlays which some consider a corner case (I don't
understand that argument, but that's another thing)):

When the package manager looks at a package, it first reads the
package's ebuild directory and gets the mtimes. It does the same for the
cache entries and validates the caches (there is more stuff in here,
like checking eclasses and so on).
Then the following happens based on the "solution" we choose:
eapi-in-filename: the package manager starts from the highest version
with a supported eapi (the others are inexistant with the used glob).
For that ebuild it reads the cache entry and decides whether or not it
can be used. If not, it proceeds to the next version, if yes, it's done.
eapi-in-ebuild: the package manager reads all cache entries and sorts
out those with an EAPI it doesn't support. The rest gets ordered and the
same procedure as above applies.

So, one of the main differences is: "reading one cache file" (if running
unstable you can asssume you support the highest version, thus reading
only one cache file) vs. "reading all cache files".

I did some performance measurements based on that. I have 1507 installed
packages with 5541 different versions/revisions.

Reading from hot cache:
1507 files: ~50ms
5541 files: ~170ms

Reading from cold cache:
1507 files: ~2.8s
5541 files: ~6s

I made a lot of assumptions here (neglecting seek between ebuild-dir and
metadata-dir, other processes using the drive, 80 ebuilds from overlays
where the ebuild would have to be read, etc.). But estimating from the
numbers above I'd say that a "emerge -uD world"/"paludis -i world" will
be at least twice as slow, which I think is not acceptable.

And I also don't understand your point of stating it's "bad design". I
mean: when coding you should "not optimize prematurely", but with
eapi-in-ebuild it is against the other principle of "not pessimize
prematurely" (Sutter/Alexandrescu: C++ Coding Standards).



-- 
Tiziano Müller
Gentoo Linux Developer, Council Member
Areas of responsibility:
  Samba, PostgreSQL, CPP, Python, sysadmin, GLEP Editor
E-Mail   : dev-zero@g.o
GnuPG FP : F327 283A E769 2E36 18D5  4DE2 1B05 6A63 AE9C 1E30