Re: [gentoo-dev] Next council meeting on 7 Dec 2009 at 1900UTC

[Brian Harring]

On Thu, Nov 26, 2009 at 04:46:49PM +0000, Ciaran McCreesh wrote:
> On Thu, 26 Nov 2009 08:33:03 -0800
> Brian Harring <ferringb@×××××.com> wrote:
> > > "Provide proof that all existing and future caches that would rely
> > > upon this validation mechanism are functions purely and exclusively
> > > dependent upon the VDB content, and I shall be happy to make the
> > > change."
> > The timestamp updating is for whenever the vdb content (addition of a
> > pkg, pkgmoves being applied, removal of a pkg, modification of
> > metadata, etc) is changed.  That's all that timestamp is for.  Vdb
> > content.
> >
> > In light of what the timestamp is, your demand for proof is pretty
> > off the mark.  If you still consider it to be a valid question,
> > please rephrase it and clarify why exactly proof must be provided
> > that people reading that timestamp (which is for vdb content only)
> > will only be using that timestamp for vdb content.
> >
> > Your request is akin to demanding proof that a hammer only be used as
> > a hammer.  It's a fricking hammer- it has one use, one way of being
> > used.  If someone goes out of their way to be an idiot, they're an
> > idiot, not the specs problem.
> >
> > Seriously, if you're actually worrying about some specific usage
> > case, state it- on the face of it, your request for proof right now
> > makes zero sense.  Kindly provide a scenario or elucidation.
>
> You're asking for a cache validation mechanism that's based not upon
> what it logically invalidates, but upon something you assume to be
> equivalent. Specifically, you assume that "VDB has changed" and "the
> installed packages have changed" are exactly the same thing, and you're
> wanting to build a cache based upon that highly questionable
> assumption. There are at least two logically different sets of
> 'changes' that might apply to VDB (metadata changes, and package
> version changes), and you're attempting to confuse the two, along with
> any others that people come up with later on.
>
> What's wrong with, instead, having cache files for "something has
> changed", "the set of installed packages has potentially changed", "the
> metadata for installed packages has potentially changed" and "the set of
> installable packages has potentially changed"? That way you can write
> your cache checks to depend explicitly upon the thing upon which they
> depend (along with a global catch-all to make future new validation
> mechanisms easier), not upon something you assume is equivalent but
> probably isn't.

Ah... there we go, you're again asking for specific timestamps such 
that specific caches can be invalidated.  Same as I said in the bug, 
you want it, propose it.  As you stated above, *still* a global 
timestamp of some sort is needed.

Seriously- if you want some specific cache timestamp, go nuts.  The 
global timestamp is still needed and that's the only one I give a damn 
about at this juncture- I'm not as much interested in layering in new 
hacky caches on the vdb to try and make it performant as I'm 
interested in building flat out, a new vdb that is designed from the 
ground up for efficiency/performance.

When the old vdb format has the timestamp requirements, I can use that 
to keep the two in sync (maintaining compatibility while being free 
to start developing a better vdb, or alternate implementations- say 
remote).  Beyond that, for people less ambitious it serves as a 
timestamp they can use for updating their own caches- not the most 
fine grained admittedly, but it's also a rare scenario.

Either way, you want specific cache timestamps, it's an addition to 
this proposal- for example I'm specifically disinterested in adding a 
pkg names cache because the gain from it isn't particularly high for 
the scenarios I'm looking at (provides/use/iuse/depends/rdepends per 
node being higher cost in my profilings).  Admittedly it speeds up 
simple lookups in the vdb of "what version do I have installed?", but 
most folk do a pmerge/emerge -Dup for that (meaning full metadata 
access still).

~harring