Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)

[Anthony G. Basile]

On 09/06/14 10:04, Ciaran McCreesh wrote:
> On Sat, 06 Sep 2014 09:51:58 -0400
> "Anthony G. Basile" <blueness@g.o> wrote:
>>> Paludis doesn't do this (and historically, Portage didn't either).
>>> We store USE etc. This is useful because it allows us to detect when
>>> people have been mucking around with DEPEND and the like.
>> This was a suggestion from mgorny and I trust his opinion on the
>> matter.  It does make sense to have metadata catche which is
>> conditionally evaluated be exported.
> Well what are you planning to use it for? Are you really suggesting
> that people are going to implement EAPI-aware, compliant dependency
> parsers that can't figure out conditionals?

No, but the resolution of the conditionals may change over time and it 
would be nice to have the information as it was at the time of the 
build.  However, this point is minor.  I'm certain something can be 
worked out among the PMS people as to what is best here.  After all, the 
request did come from the portage camp.

>
>> 1) This cannot be utterly arbitrary because there are utilities and
>> portions of portages code which does make use of precisely this
>> information.
> What Portage does internally is up to Portage. What we don't want is to
> encourage external utilities that are utterly inflexible and that make
> strange assumptions.

Neither do we want to hamper what developers might want to do.  The 
authors of sys-apps/elfix, app-portage/gentoolkit, 
app-portage/portage-utils, app-portage/eix are not "utterly inflexible" 
utilities makeing "strange assumptions".  They are useful utilities as 
anyone following this thread would agree.  Their assumption would go 
from being non-standard to standard with the acceptance of this GLEP.  
They would then operate with both portage and paludis.

>
>> 2) With the exception of some embedded systems where everything is
>> statically linked, all modern systems have dynamic linking.  And all
>> dynamic linking has information in its objects which associates the
>> executables with the library they link against.  This is the
>> essential information to be stored.
> Which isn't what you're asking for... You're asking for it in a
> particular format.

I will incorporate better language which goes to the heart of what's 
needed and makes the ELF quantities an example rather than the demanded 
format.  In other words, I will remove the ELF bias.  Since dynamic 
linking information is generated at build time, is expensive to 
regenerate and is useful to other utilities, it falls under the scope of 
the GLEP.  Of course we wish to extend this to any executable format.  
We focus on ELF because of its familiarity, but not at the exclusion of 
others.

>
>> 6) Without a standard here, we have utilites which make use of this
>> cached information in the tree which only work with portage but not
>> paludis.  This problem can be solved by removing those utilities,
>> which is undesireable, by standardizing what needs to be exported
>> from the PM, or by living with the status quo which is having useful
>> packages in the tree which don't work with paludis.
> The solution is to replace those utilities with something that works in
> proper generality.

You can do this, but it would be terribly inefficient to regenerate 
expensive information already generated by PM.  For example, `equery` 
from gentoolkit allows the user to gather all sorts of information about 
installed packages, eg. "What package does this file belongs to?"  A 
very natural question.  Without this information cached in portage's 
VDB, how would equery do this?  It would  have to rebuild package by 
package until it finds one that gives the file we're looking for?!  This 
is absurd.  Rather gentoolkit opens up /var/db/pkg and read the 
information out of there.

For gentoolkit to work "in proper generality" we would need PM's to 
export this information by some common API, so there is a generality.  
That's what GLEP 64 is about.  You point argues in favor of the GLEP.


>
>>> You've also not discussed how this interacts with Portage's
>>> package.provided misfeature.
>>>
>>> Finally, you don't have any way of using this information, since you
>>> don't have a way of knowing what packages are installed.
>> I don't understand your reasoning behind these points, can you please
>> explain.
> Well you can't ask for information about packages unless you know
> what's installed, and you haven't asked for a general API for that.

Ah, okay.  That should be added explicitly since its only implied right 
now.  Thanks.

>
> And when you do ask, is a package that's "provided" installed, and if
> so, what's its metadata?
>

When the package is installed, that data should have been cached.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@g.o
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA