Re: [gentoo-portage-dev] Current portage well designed, but badly used

[Gregorio Guidi]

On Wednesday 01 December 2004 13:08, Brian Harring wrote:
> > > 1) don't modify $portdir/metadata/cache.  Modifying the rsync'd cache
> > > directly results in more to rsync for the next sync.  Going that route
> > > would result in a lot of dial up users out for blood.
> > > 2) the user may not be using the same cache backend as the distributed
> > > cache- the rsync'd cache is portage_db_flat, while the user may be
> > > using a custom sqlite backend.  I suspect this will be come more common
> > > place whenever cvs becomes stabled- the cache backend is being
> > > refactored currently.
> > > 3) cleanse stale entries from the cache, so you don't end up w/ a
> > > couple thousand extra files sitting in your local cache.  Stable
> > > portage accomplishes this by wiping *all* local cache entries, and
> > > transferring the *entire* rsync'd cache over.  Cvs is smarter,
> > > transfers only what has changed, and removes the stale entries (this is
> > > why it's faster).

> > With respect to point 3, the solution you imlpemented is surely the right
> > thing to do.
> > With respect to point 1, maybe a good solution could be to just check
> > cache validity in $portdir/metadata when the cache is needed, and write a
> > new cache in /var/cache/edb if it is not valid.
>
> Err... elaborate
> Assuming I'm following, you're proposing doing the updates to local
> cache, and reading from the metacache?  If so, that's twice the level of
> potential stats/reads required, which won't speed it up much :)
> ~brian

Nah, it would be something like:
- see if cache exists in /var/cache/edb (one cheap access() call)
 - if it exists, validate (costly stat() calls, but 99% success)
 - if it does not exists or is not valid, read cache in $portdir/metadata and   
   validate (stat() calls, 99% success).
 - if cache is not valid, create new cache in /var/cache/edb (ebuild  
  sourcing: takes a lot of times).
- read cache.

- after emerge sync, go through cache files in /var/cache/edb (usually not 
  many) and remove entries corresponding to deleted ebuilds.

But I'm not aware of portage internals, for instance, I'm thinking of a system 
where the cache is read only one time and then stored in memory.
Does portage does so, or does it read the cache every time a key is needed 
from there?

Gregorio

--
gentoo-portage-dev@g.o mailing list