Gentoo Archives: gentoo-performance

From: Jesse Guardiani <jesse@×××××××.net>
To: gentoo-performance@l.g.o
Subject: [gentoo-performance] Re: Re: portage performance
Date: Fri, 30 Jul 2004 14:13:02
Message-Id: cedl15$hqs$
In Reply to: Re: [gentoo-performance] Re: portage performance by Brian Harring
Brian Harring wrote:

> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > >> The basic problem in searching is actually that it isn't implemented >> smartly >> in current portage. I have working (emerge -s like) code that is >> blazingly >> fast as it does not actually open all ebuilds. > Searching works off of the cache for the most part, if a cache entry is > stale, it's updated (eg the ebuild is opened and srced). > Unless you're not checking the cache and updating it as you proceed, > you're implementation ought to suffer the same limitation. > >> Doing description searching is >> impossible to do fast without some kind of cache. I don't think >> creating a >> reliable cache for that is going to be a priority, > There are 2 things that need to be done (in my books at least) to step > up the speed of a description search- > A) sql based cache backend, whether sqlite or mysql.
If it comes to this, I definately vote for sqlite. It's much faster than mysql for embedded apps, and you don't have to worry about whether or not the server is running.
> Either that, or > extend the flat cache to store the descriptions in a central index.
I think this is a good idea. AFAIK, this is what FreeBSD does. Perhaps tinycdb could be used? (OCaml has a CDB module available too. You can get it here: (BSD licence))
> B) alter the search description alg so that instead of stepping through > each entry getting the description, we just state "give me all packages > that have a description matching blar", and leave it up to the backend > to decide what is the most efficient way to search. With flat cache, > we'd still have to go file by file; w/ a sql variant, it could take > advantage of the appropriate syntax.
I personally prefer well designed hash table schemes. This is part of a base operating system, so it needs to be extremely efficient. IMO, SQL databases tend to encourage laziness. This is not to say that they don't have their place, but I don't think they belong in a package management system. Still, it's a novel idea, I think. And offers a great deal of flexibility. -- Jesse Guardiani, Systems Administrator WingNET Internet Services, P.O. Box 2605 // Cleveland, TN 37320-2605 423-559-LINK (v) 423-559-5145 (f) -- gentoo-performance@g.o mailing list