Re: [gentoo-dev] RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.

[Anthony G. Basile]

On 08/31/14 03:26, Micha³ Górny wrote:
> Dnia 2014-08-30, o godz. 16:02:51
> "Anthony G. Basile" <blueness@g.o> napisa³(a):
>
>> I've written a GLEP which outlines a standard for what information
>> should be stored by any package management systems (PMS) in /var/db/pkg
>> (VDB) and mandates some API for exporting it to other tools [1].
> I have trouble understanding the goal. As far as I can see, the idea
> here is that every PM stores all information in any format, and exports
> a Python API that has any synopsis and gives access to it... in any
> format.
>
Not exactly.  The point is to *standardize* what is meant by "all 
information" so that all package managers export the same minimum set of 
information.  The most important being NEEDED.ELF.2 which is portage's 
VDB but not paludis.  Also, it can be in any format and exported in any 
way so long as it is well documented.  The goal is to have tools other 
than PM's make use of this information.  The example that began this is 
revdep-pax which uses NEEDED.ELF.2 to trace out linking so as to migrate 
PaX flags between ELF objects.

> Wouldn't it be better to at least agree on some API for the metadata
> exports? That will spare us the necessity of wrapping them all
> in a common package or in every tool itself. Maybe it could even bring
> some degree of interoperability between package managers.
Yes, I agree completely, but I didn't feel I was mandated to do so, 
which is what leaves me unsatisfied with what I wrote and reflected in 
your first paragraph.  However, I didn't want to bind the hands of the 
people writing the PM's either.  I wanted to make their task as easy as 
possible.  So long as all PM's export the same *minimum* set, developers 
writing non-PM tools which depend on VDB information can know what to 
expect and how to get at it.  Since NEEDED.ELF.2 is the critical for the 
reasons stated in the GLEP, and its not cached by some PM's, it was the 
subject of focus during the Council Meeting.

Having said that, I very much would like to say what that API looks 
like.  What you have below works for me.

>
> As for the spec itself:
>
> 1. You're missing some of the metadata variables (RESTRICT,
> PROPERTIES...). Wouldn't it be better to make one point worded like
> 'ebuild metadata as listed in PMS 13.2 Cache File Format'?

The point of the GLEP is to define a *minimum* set of exported metadata 
information, so I thought about what that minimum set should be.  I 
should have cleaned stuff up more before handing the draft over to 
creffett but I mistakingly assumed I could continue editing the wiki page.

Anyhow, now is the time for people to suggest what that minimum set 
should be.  I'll look at PMS 13.2 Cache File Format.

>
> 2. For *DEPEND, REQUIRED_USE (another one you missed) PMs store
> dependency trees with USE conditionals evaluated. You may want to
> explicitly note that.

Noted.

>
> 3. I would use a copy of ebuild environment variables at the time of
> completing the build (leaving last src_* phase?) -- IOW,
> environment.bz2.

Noted.

>
> 4. BUILD_TIME is not defined anywhere, so you may want to replace that
> with verbose explanation of what is to be stored. For the remaining
> metadata, you may want to reference PMS (the specification).

Noted.

> 5. Please do not recommend Python modules since it discriminates
> package managers written in C flavors.

What's wrong with C-python bindings?  Or anything-anything bindings.  
I'm not discriminating against flavors of PM's.  A PM can export this 
information via a C library, as long as the API is documented.  Its up 
to the developers of utilities consuming this information to "hook" in.

Having said that, I like what you have below and I'd be happy to work on 
it.  I can already see how to proceed with portage.

> Instead, I suggest a plain CLI
> API that gives best portability possibly. Alike:
>
>    $ query-installed metadata sys-apps/coreutils-8.23 RDEPEND SLOT
>    ...
>    0
>
>    $ query-installed file sys-apps/coreutils-8.23 /usr/bin/timeout SONAME
>    ...
>
> It should also have batch interface for querying multiple packages
> quickly -- passing requests via stdin:
>
>    $ query-installed batch
>    [>] metadata sys-apps/coreutils-8.23 RDEPEND SLOT
>    [<] ...
>    [<] 0
>    [>] file sys-apps/coreutils-8.23 /usr/bin/timeout SONAME
>    [<] ...
>
> 6. Your Portage snippet uses outdated API. The modern one is to use
> create_trees().

You mean the |vardb = portage.db[portage.root]['vartree'].dbapi stuff?  
We can talk about this later.|



-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@g.o
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA