Gentoo Archives: gentoo-dev

From: Stuart Herbert <stuart@××××××××××××××××××.uk>
To: gentoo-dev@g.o
Subject: [gentoo-dev] FW: rsync etiquette follow-up
Date: Mon, 12 May 2003 16:01:58
Message-Id: 010601c3189f$152fb2a0$c000a8c0@Churchill
1 Dear all,
2
3 I originally sent the comments below to GWN as feedback on the recent call
4 for rsync etiquette. Kurt Lieber [klieber@g.o] asked me to forward
5 on to this list, for further discussion.
6
7 Please be gentle - this is my first posting to the list ;-)
8
9 Best regards,
10 Stu
11 --
12
13 -----Original Message-----
14 From: Stuart Herbert [mailto:stuart@××××.org]
15 Sent: 09 May 2003 11:24
16 To: 'gwn-feedback@g.o'
17 Cc: 'stuart@××××.org'
18 Subject: rsync etiquette follow-up
19
20
21 Hi there,
22
23 I read with great interest your article on the need for rsync etiquette.
24 I'm sure lots of people have already written to you making the same point.
25 I'd like to add my voice to this point.
26
27 Gentoo's 'emerge' is a wonderful tool in many ways - but when it comes to a
28 site containing multiple machines, improvements in its design could help
29 reduce the load on your rsync servers.
30
31 I run four Gentoo boxes at the moment, and they're all connected to the
32 Internet through a single firewall configured with masquerading. If I were
33 to rsync each machine just once a day, I wonder whether it would look like
34 one machine had rsync'd four times from the point of view of the Gentoo
35 rsync mirror that I use?
36
37 Anyway, whenever I install a new package onto a machine, it is important to
38 my work that *each* of these machines has the same version of the package.
39 The machines are different x86 architectures, so building a binary package
40 on the one machine isn't my preferred choice. The whole point of using
41 Gentoo is that each machine runs code that is specifically optimised for the
42 hardware.
43
44 To achieve this, there's no getting away from it. I have to 'emerge rsync'
45 on each machine, and 'emerge <package>'. The 'emerge rsync' is there to
46 ensure that each machine picks up the same version of both the package, and
47 its dependencies. This can be done via cron once every couple of days. But
48 it still means that I end up hitting your rsync mirrors *once* for every
49 Gentoo machine I run. I also end up hitting your distfile mirrors *once*
50 per machine as well.
51
52 I know that I could run an rsync mirror just for internal use - and that
53 would help a lot. Running a distfiles mirror is a lot less practical. It
54 would be much better if there was a way to share '/usr/portage' across
55 multiple machines. You can't do this safely via NFS. If two machines try
56 to download the same distfile at the same time, they interfere with each
57 other.
58
59 It'd be much better if emerge could be changed to a client/server model,
60 where the emerge command becomes a client that contacts the (possibly
61 remote) emerge server to do all the downloading and rsyncing. For people
62 running both client/server on the same machine, there's no noticable
63 difference. For people like me, trying to run a site of machines, I'm able
64 to reduce the amount of load I place on your rsync servers and your distfile
65 servers.
66
67 This gives me another advantage. I don't have to have a (potentially large)
68 /usr/portage/distfiles on each machine. If three of my machines are
69 clients, and the server only runs on the fourth machine, I can setup a cron
70 job to clear out /usr/portage/distfiles on each of the clients on a nightly
71 basis - and keep my diskpage usage down. (As an aside, it'd be great to see
72 /usr/portage moved into /var. One of my few true disappointments with
73 Gentoo is having to have /usr mounted read-write a lot of the time)
74
75 I hope that I've explained it right - and that some intrepid emerge
76 developer will take on the task. And soon.
77
78 Best regards,
79 Stu
80 --
81
82
83
84 --
85 gentoo-dev@g.o mailing list

Replies

Subject Author
Re: [gentoo-dev] FW: rsync etiquette follow-up Priit Laes <amd@××.ee>
Re: [gentoo-dev] FW: rsync etiquette follow-up Steven Lucy <slucy@××××××××.edu>
Re: [gentoo-dev] FW: rsync etiquette follow-up Sven Vermeulen <swift@g.o>
Re: [gentoo-dev] FW: rsync etiquette follow-up "Robin H.Johnson" <robbat2@g.o>