Gentoo Archives: gentoo-portage-dev

From: Brian Harring <ferringb@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] Cache rewrite backport
Date: Wed, 12 Oct 2005 03:53:03
Message-Id: 20051012035232.GA8851@nightcrawler
In Reply to: Re: [gentoo-portage-dev] Cache rewrite backport by Bastian Balthazar Bux
1 On Wed, Oct 12, 2005 at 03:49:44AM +0200, Bastian Balthazar Bux wrote:
2 > Brian Harring ha scritto:
3 > > On Wed, Oct 12, 2005 at 12:01:12AM +0200, Bastian Balthazar Bux wrote:
4 > >
5 > >>Sorry, but here the results are not those expected:
6 > >
7 > > .51.22 vs .53_rc5... try with a vanilla .53_rc5 please
8 >
9 > here they are, also added a test with a dirty trick to precharge the
10 > portage dir and see what happen. Look like there is a small improvement.
11 > Now it's late.
12 >
13 > ==== time emerge --metadata; 1st run; 2.0.53_rc5 vanilla
14 > real 9m44.449s
15 > user 4m51.034s
16 > sys 0m24.754s
17 >
18 > ==== time emerge --metadata; 2nd run; 2.0.53_rc5 vanilla
19 > real 2m50.932s
20 > user 0m12.597s
21 > sys 0m3.836s
22 >
23 > ==== time emerge --metadata; 3rd run; 2.0.53_rc5 vanilla
24 > real 1m55.445s
25 > user 0m12.501s
26 > sys 0m3.416s
27 >
28 > ==== tar -c /usr/portage/* >/dev/null & time emerge --metadata
29 > ==== ; 4th run; 2.0.53_rc5 vanilla
30 > real 1m10.275s
31 > user 0m13.377s
32 > sys 0m4.740s
33
34 I'd wonder how this ^^^ compares to round 5 of the backport patch-
35 (attached). It's a bit further then what's in 3x, although assuming
36 it behaves well I'll merge that into the rewrite also.
37
38 > ==== tar -c /usr/portage/* >/dev/null & time emerge --metadata
39 > ==== ; 4th run; 2.0.53_rc5 patched
40 > real 4m12.131s
41 > user 0m13.661s
42 > sys 0m10.329s
43
44 Difference is due to the fact the rewrite's metadata->local cache is
45 cloning code (think cp) rather then updating...
46
47 Forgot about that, wrote that code a while back you see :)
48
49 Made some somewhat tricky changes to the code; stable's best case is
50 2 stats, 2 reads, 2 opens, and 2 closes per ebuild. Worst case is a
51 write thrown in, plus utime.
52
53 Best case with this patch is 2 stats, worst is still the same but
54 occurs only when eclasses change.
55
56 Alec and zach tested it fair bit, sounds of it it's performing fairly
57 well. Would be curious about stats from those running truly crap
58 hardware, 20m metadata transfer for example.
59
60 Forewarning, if you're switching between the patch and .53_rc3, you'll
61 get some screwwy stats on the first run since it's a cache
62 invalidation, eg wipe the cache via an rm -rf. :)
63 ~harring

Attachments

File name MIME type
3.0-cache-backport-experimental-4.patch.bz2 application/octet-stream

Replies

Subject Author
Re: [gentoo-portage-dev] Cache rewrite backport Brian Harring <ferringb@g.o>