Re: [gentoo-dev] Re: How not to discuss

[Richard Freeman]

Ryan Hill wrote:
> I'm tired of playing, as I'm sure you are.  So please,
> let's be quiet now, and let the big people talk.
>

This is a public list designed to facilitate discussion of gentoo 
software development.  Anybody with something constructive to say is 
more than welcome to speak up - particularly gentoo staff.

I don't pretend to be an expert on package management.  However, hiding 
internal implementation details is just good design.  I can see how 
putting eapi in the filename can be a convenience to the package 
manager, but it still seems like a bad design, as it exposes end users 
to an implementation detail of the package management system.

There are lots of ways that EAPI could be cached that would avoid the 
various penalties that have been referred to.  Even without an improved 
cache the penalty seems superior to accepting the design compromise of 
EAPI in the filename.

As to how EAPI could be cached goes - I could think of a few high-level 
design options:

1.  Cache files are distributed with the portage tree.  EAPIs that break 
the cache format would use different files that older package managers 
would ignore.  Downside is that it doesn't handle user-modified ebuilds 
(unless the user tells the package manager to regenerate the cache), and 
it doesn't handle overlays unless the maintainer generates the cache.

2. Cache files are generated when the tree is synced.  The package 
manager would look at the list of modified files and scan only those 
files one time to index them.  The index could contain the mtime and 
path of the file.  Then, when you perform an operation the package 
manager could check the mtimes in the directories containing those files 
and see if anything was touched and regenerate the cache if needed. 
This takes a little more time during syncing but I suspect that it would 
perform very well - after all after a sync all those files would be in 
the disk cache anyway.  A suitably clever package manager could read the 
files as they are being synced and guarantee they are in-memory.

If we were talking about a 300TB table that got 300k transactions per 
second I could see why we'd be talking about hacks to sacrifice 
normalized design for speed.  We're talking about a package database - 
one that contains < 150k records.  Sacrificing good design for speed 
(instead of improving the algorithm) is a short term gain for a 
long-term cost.