Re: [gentoo-dev] Re: VDB access

[Ciaran McCreesh]

On Tue, 5 Feb 2008 08:43:55 +0100
Christian Faulhammer <opfer@g.o> wrote:
> > This will let package managers use a format other than VDB. A well
> > designed replacement can shave a minute off cold cache command
> > times.
>
>  How much gain can be expected?

There're two common operations that're really really slow (because they
need lots of filesystem access) using VDB: turning a pkg into a cat/pkg
and finding all PROVIDEd packages. A redesigned format can, for example,
make the former a single filesystem operation and the latter a single
directory read.

If you're looking for numbers, you can see very roughly how long a VDB
load takes off cold cache using:

echo 2 | sudo dd of=/proc/sys/vm/drop_caches
find /var/db/pkg/ -type f -not -name 'environment.bz2' \
    | xargs cat >/dev/null

A different on-disk format could avoid 90% of those filesystem accesses
for many operations.

There's a more subtle issue with VDB and the scope of locking required
by multithreaded implementations. This one's not an issue for
interpreted languages, but it's a minor nuisance in places where
there's no language-induced locking.

> And what package managers will allow native_built_with_use?

I doubt Portage will, since if someone's going to make changes to
Portage that'd let it use something other than VDB, they've clearly got
enough time to implement use deps as a side project, which would make
built_with_use obsolete. Paludis would certainly implement it -- we've
already got a redesigned on-disk db format that we use for tracking
unpackaged packages, and repurposing that to handle ebuilds wouldn't be
very hard.

Incidentally, if anyone is still thinking that built_with_use is
fine... It doesn't handle IUSE defaults, which means package managers
have to store a hacked version of IUSE in VDB for EAPI 1. This isn't
documented anywhere and is highly unobvious.

-- 
Ciaran McCreesh