Re: [gentoo-portage-dev] SQLite backend?

[Brian Harring]

On Tue, Jan 17, 2006 at 02:01:29AM -0200, Gustavo Sverzut Barbieri wrote:
> Hello,
>
> I admit I have not followed last threads about cache and new
> infrastructure (plugins and stuff).

Might suggest you take a look at the cache rewrite- it already has a 
sqlite backend in it, although that's not coded against pysqlite2 
(thus it needs some param work).

> However I followed the template and coded a SQLite3 (pysqlite2)
> backend: http://www.gustavobarbieri.com.br/gentoo/portage_db_sqlite.py

Really should fix the 403 if you're looking for comments.

Guessing this is against stable... in which case I'd point you at the 
2.1 line, since the stable cache is now dead from a release standpoint 
(it'll exist in 2.0.*, but everything beyond is the new cache).

> The main problem is that it's really slow to search and calculare
> dependencies. Could some developer take a look and tell me where I can
> improve it?

That's due to portage access patterns- select should be slower then 
flat_list/flat_hash, and the way portage pulls from the db (namely, 
key by key) doesn't work incredibly well for an rdbms backend- 
basically, the api is such that instead of doing a single select, 
it'll do multiple.

> In an ideal world, I wouldn't have to pickle/unpickle data but have
> its data in columns instead... one would re-encode the version strings
> using a pattern that is comparable by string, so you should be able to
> search for a given string or higher versions.

Cache rewrite is intended to work towards that- still requires the 
restriction framework I've got in savior, but gradually working 
towards that.

~harring