Gentoo Archives: gentoo-soc

From: Zac Medico <zmedico@g.o>
To: gentoo-soc@l.g.o
Subject: Re: [gentoo-soc] GSoC - cache sync/self-contained ebuilds
Date: Thu, 24 Mar 2011 16:58:26
Message-Id: 4D8B784E.9080104@gentoo.org
In Reply to: Re: [gentoo-soc] GSoC - cache sync/self-contained ebuilds by Michael Seifert
1 On 03/23/2011 02:01 PM, Michael Seifert wrote:
2 > -----BEGIN PGP SIGNED MESSAGE-----
3 > Hash: SHA1
4 >
5 > Am 23.03.2011 20:47, schrieb Donnie Berkholz:
6 >> On 14:43 Wed 23 Mar , Rich Freeman wrote:
7 >>> On Wed, Mar 23, 2011 at 1:44 PM, Michael Seifert
8 >>> <michael.seifert@×××.net> wrote:
9 >>>> I cannot tell you at this stage, sorry. The real loss for steps 3/4
10 >>>> has to be measured after the implementation. The problem with the
11 >>>> estimates here is that the assembled ebuilds also contain the
12 >>>> sources and the eclasses. Maybe I will do some number crunching on a
13 >>>> few selected ebuilds.
14 >>>
15 >>> A more critical factor could be the dependencies - unless we otherwise
16 >>> cache them. To install a package you need to walk the dependency tree
17 >>> (well, at least until you hit installed packages with the right USE
18 >>> flags). That requires one set of fetches for each level you traverse,
19 >>> and that means at least one round trip per level.
20 >
21 > You are right. I only considered the size improvements, not the changes
22 > to speed.
23 > Anyway, is the calculation of dependencies handled differently in
24 > current portage versions?
25
26 Well, it would be inefficient to open separate TCP connections for
27 individual metadata files since there are so many of them and they are
28 so small. This is why package managers typically download the metadata
29 for all packages as a single bundle. For example, see the type of
30 metadata bundle that is used to implement PORTAGE_BINHOST support:
31
32 http://tinderbox.dev.gentoo.org/default-linux/x86/Packages
33
34 It's conceivable that you could simply use rsync to sync the
35 metadata/cache/ subdirectory from
36 rsync://rsync.gentoo.org/gentoo-portage/. However, since the rsync tree
37 constantly mutates and doesn't provide any kind version control, it
38 would not be very practical to use it in this way. If you fetch the
39 metadata and the ebuilds separately, you need a way to guarantee that
40 you can fetch exactly the same revisions of ebuilds that the earlier
41 fetched metadata corresponds to.
42 --
43 Thanks,
44 Zac

Replies

Subject Author
Re: [gentoo-soc] GSoC - cache sync/self-contained ebuilds Michael Seifert <michael.seifert@×××.net>