Re: [gentoo-portage-dev] Current portage well designed, but badly used

[Brian Harring]

On Wed, 2004-12-01 at 01:37, Gregorio Guidi wrote:
> On Tue, 30 Nov 2004 06:22:07 -0800, Brian Harring <ferringb@g.o>
>
> > If you're referencing the nice long pause after sync'ing, that's
> > transfer of the cache from ${PORTDIR}/metadata/cache to
> > /var/cache/edb/dep/${PORTDIR}; that's been speed up also.  Potentially
> > might have that pause eliminated also, although that would require
> > ensuring the tree is readonly- that's another can of worms.
>
> I was just thinkng about that, and I could not fully understand this point.
> Could you explain a bit more?
Expanding on the commentary earlier about a readonly tree/cache vs the
current- rsync'd trees are currently treated as modifiable by the user. 
In other words, the cache for the tree must be verified- this means
pulling the mtime from the ebuild, and comparing it to the stored mtime
in the cache.  Additionally, if the ebuild inherits eclasses, need to
check the eclass's mtime and the stored mtime.  All of this is done to
ensure that if the eclass/ebuild changes, the cache is accurate.

So... the rsync'd cache ($portdir/metadata/cache) is accurate to the
rsync'd tree, minus user modifications- since the tree is treated as
modifiable, the cache *can potentially* be updated.

That nice lil long pause is portage transfering the cache from
portdir/metadata/cache, into the users local cache
(/var/cache/edb/dep/$portdir typically).  This is done for a few
reasons,

1) don't modify $portdir/metadata/cache.  Modifying the rsync'd cache
directly results in more to rsync for the next sync.  Going that route
would result in a lot of dial up users out for blood.
2) the user may not be using the same cache backend as the distributed
cache- the rsync'd cache is portage_db_flat, while the user may be using
a custom sqlite backend.  I suspect this will be come more common place
whenever cvs becomes stabled- the cache backend is being refactored
currently.
3) cleanse stale entries from the cache, so you don't end up w/ a couple
thousand extra files sitting in your local cache.  Stable portage
accomplishes this by wiping *all* local cache entries, and transferring
the *entire* rsync'd cache over.  Cvs is smarter, transfers only what
has changed, and removes the stale entries (this is why it's faster).

So... that's why portage basically copies the cache to a different
location.  If the tree were treated as readonly, and it was enforced in
some manner, that transfer wouldn't be needed.  This is something that's
being batted around also- the code for making the cache readonly is
already done fex.
~brian


--
gentoo-portage-dev@g.o mailing list