Gentoo Archives: gentoo-portage-dev

From: Sandy McArthur <sandymac@g.o>
To: gentoo-portage-dev@g.o
Subject: [gentoo-portage-dev] portage-ng wish list
Date: Fri, 05 Dec 2003 16:22:36
Message-Id: 3FD0FFA9.5060009@gentoo.org
In Reply to: [gentoo-portage-dev] portage-ng design competition -- not yet by Daniel Robbins
1 Daniel Robbins wrote:
2 > focus on the capabilities you want in Portage first. Tell us about
3 > these. These need to be documented first.
4
5 Well, I think I'd like to see implemented:
6
7 1. Better mirror detection. Currenty portage uses hard coded data with a
8 touch of randomness either from DNS RR or via picking one choice from a
9 list. Little is done to pick the best mirror for the user.
10
11 I think portage should query for a current list of mirrors using DNS
12 Service Discovery (DNS-SD) and perform some simple connectectiviy tests
13 to find the "best" mirror. After portage has discovered the "best"
14 mirror it constructs a url as needed, be it rsync or wget. A reasonable
15 caching method for the connectectivity tests would be good too.
16
17 DNS-SD is currently an internet standard draft which is used by Zeroconf
18 but it doesn't depend on Zeroconf. It is well thought out and can
19 support things like path components which would be needed for a distfile
20 mirror since that is more than hostname and a fixed path.
21
22 More info at: http://www.dns-sd.org/ and
23 http://files.dns-sd.org/draft-cheshire-dnsext-dns-sd.txt
24
25
26 2. Moving the default location for distfiles from /usr/portage/distfiles
27 to /var/cache/distfiles . Write access to /usr should only be needed
28 when installing a new package. I should also be able to export a NFS
29 mount of a portage tree readonly and share it across a cluster.
30
31
32 ------------
33 Below here are wild ideas that probably need more baking but I'll throw
34 them out there anyway.
35
36
37 3. Make it so the portage tree is sparse until part of it is requested.
38 There are thousands of packages in the portage tree and I use maybe a
39 few hundred. Why can't I enable an online mode that only rsyncs the
40 herds and empty package dirs until I try to `emerge herd/foo`. Then it
41 rsyncs the package fetching the ebuilds and related files as they are
42 needed. While this would make many times more connections to rsync
43 mirrors the amount of bandwidth would be a lot less because if I'm a KDE
44 user and don't touch Gnome then when changes happen to Gnome packages I
45 don't have to use bandwidth to sync them as I don't care about each
46 little update to packages I haven't installed.
47
48
49 4. Support Zero Install as a portage mirroring system. Zero Install
50 seems to be httpfs but doesn't require anything special about the web
51 server so that we could take advanage of existing web servers as
52 mirrors. More info at: http://zero-install.sourceforge.net/
53
54
55 5. Do something to speed up the long 'updating portage cache' step in a
56 `emerge sync`.
57
58
59 6. Support bittorrent distfile distrobution.
60
61
62 Sandy McArthur
63
64
65 --
66 gentoo-portage-dev@g.o mailing list