Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-performance
Navigation:
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: gentoo-performance@g.o
From: Paul de Vrieze <pauldv@g.o>
Subject: Re: Re: portage performance
Date: Tue, 3 Aug 2004 10:49:17 +0200
On Tuesday 27 July 2004 00:57, Brian Harring wrote:
> > The basic problem in searching is actually that it isn't implemented
> > smartly
> > in current portage. I have working (emerge -s like) code that is
> > blazingly
> > fast as it does not actually open all ebuilds.
>
> Searching works off of the cache for the most part, if a cache entry is
> stale, it's updated (eg the ebuild is opened and srced).
> Unless you're not checking the cache and updating it as you proceed,
> you're implementation ought to suffer the same limitation.

Basically it does a directory glob selecting valid candidates. Those 
candidates are then checked whether they are real packages. If they are, they 
are valid results and returned.

> There are 2 things that need to be done (in my books at least) to step
> up the speed of a description search-
> A) sql based cache backend, whether sqlite or mysql.  Either that, or
> extend the flat cache to store the descriptions in a central index.
> B) alter the search description alg so that instead of stepping through
> each entry getting the description, we just state "give me all packages
> that have a description matching blar", and leave it up to the backend
> to decide what is the most efficient way to search.  With flat cache,
> we'd still have to go file by file; w/ a sql variant, it could take
> advantage of the appropriate syntax.

Probably some kind of caching or tool (like makewhatis) is the way to go. An 
option would be to use grep first to limit the amount of candidate packages 
that get examined for real (grep is a lot cheaper than parsing).

> Since there is code for a sql based cache backend, B has been bounced
> around in #gentoo-portage a bit.  Prior to it actually happening I
> would think the sql db code would need to be cleaned up/QA'd/etc.
>
> Course, there still is the issue of verifying that the cache entry
> isn't stale... :)

For now on I don't have any persistent caching in my working code (except 
where it uses old code for accessing current ebuilds) to keep it simple. It 
actually allready is quite fast.

> Err, eh?  If the tree is corrupted, and sync'd against a
> good/non-corrupted tree, it ought to be reverted to a sane state.

Exactly

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@g.o
Homepage: http://www.devrieze.net
Attachment:
pgpysQz22murd.pgp (signature)
References:
portage performance
-- Jesse Guardiani
Re: Re: portage performance
-- Paul de Vrieze
Re: Re: portage performance
-- Brian Harring
Navigation:
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: Re: portage performance
Next by thread:
Re: Re: portage performance
Previous by date:
Re: gentoo performance problem with quota.
Next by date:
Best Raid0 blocksize/filesystem (benchmarks included)


Updated Jun 17, 2009

Summary: Archive of the gentoo-performance mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.