Gentoo Archives: gentoo-portage-dev

From: Brian Harring <ferringb@g.o>
To: gentoo-portage-dev@l.g.o
Subject: [gentoo-portage-dev] Cache rewrite backport
Date: Tue, 11 Oct 2005 07:11:13
Message-Id: 20051011071045.GA23821@nightcrawler
1 Hola.
2
3 So... short version, for those who weren't aware, cache subsystem got
4 gutted and rewrote in >=2.1 base code.
5
6 Couple of reasons why the old design needed to be pitched
7 1) Lots of forced stat calls for no good reason.
8 2) designed around per category, which doesn't fly when you're doing a
9 remote cache- pooling being *required* for rdbms for a single
10 repository cache sucks.
11 3) Two seperate caches per repository effectively; eclass_cache (which
12 is global), and repository cache.
13
14 So... those are the main 3. Cache rewrite that's in 2.1, and 3.x
15 (latter being saner since it's actively worked on) I've backported to
16 stable, and attached to this email.
17
18 It's... kind of large, but there's a couple of good reasons for it.
19 The original cache cleansing code is included, throwing out emerge's
20 reimplementation of it (this one is a bit more extensible)- beyond
21 that, eclass_cache got ripped out of portage.py and into it's own
22 module.
23
24 Portage.py modifications are pretty minimal also.
25
26 I'd be *very* interested in timing runs for with, vs without- the
27 cache rewrite lacks a lastX cache that was used in stable's cache to
28 cover up performance issues.
29
30 Probably will need to throw in a wrapper to stable re-adding it, due
31 to the fact stable's calling patterns to the cache are bad (instead of
32 making a single request for all keys needed, makes N calls resulting
33 in N io actions rather then 1), but I'd like to get stats backing that
34 up before adding a bandaid back in.
35
36 Either way, test, feeding stats back, etc, would be appreciated ;)
37 Thanks,
38 ~harring

Attachments

File name MIME type
3.0-cache-backport.patch.bz2 application/octet-stream

Replies

Subject Author
Re: [gentoo-portage-dev] Cache rewrite backport Brian Harring <ferringb@g.o>
Re: [gentoo-portage-dev] Cache rewrite backport Brian Harring <ferringb@g.o>
Re: [gentoo-portage-dev] Cache rewrite backport Paul Varner <fuzzyray@g.o>
Re: [gentoo-portage-dev] Cache rewrite backport Thomas de Grenier de Latour <degrenier@×××××××××××.fr>