Gentoo Archives: gentoo-project

From: Brian Evans <grknight@g.o>
To: gentoo-project@l.g.o
Subject: Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method
Date: Tue, 18 Dec 2018 18:15:09
Message-Id: 7da3ce86-d7c5-0336-886d-c43a6144a5fb@gentoo.org
In Reply to: [gentoo-project] RFC: Dropping rsync as a tree distribution method by Alec Warner
1 On 12/15/2018 11:15 PM, Alec Warner wrote:
2 > Hi,
3 >
4 > I am currently embarking on a plan to redo our existing rsync[0] mirror
5 > network. The current network has aged a bit. Its likely too large and is
6 > under-maintained. I think in the ideal case we would instead pivot this
7 > project to scaling out our git mirror capabilities and slowly migrate
8 > all consumers to pulling the git tree directly. To that end, I'm looking
9 > for blockers as to why various customers cannot switch to pulling the
10 > gentoo ebuild repository from git[1] instead of rsync.
11 >
12 > So for example:
13 >
14 > - bandwidth concerns (preferably with documentation / data.)
15 > - Firewall concerns
16 > - CPU concerns (e.g. rsync is great for tiny systems?)
17 > - Disk usage for git vs rsync
18 > - Other things i have not thought of.
19 >
20 > -A
21 >
22 > [0] This excludes emerge-webrsync; which I don't plan on touching.
23 > [1] Rich talked about some downsides earlier
24 > at https://lwn.net/Articles/759539/; but while these are challenges
25 > (some fixable) they are not necessarily blockers.
26
27 I personally would be sad to see rsync go as I use the git developer
28 tree as my main repository on 2 machines. This is so I can develop and
29 update from the single source. These have no news or md5-cache and it
30 can be painful to generate metadata on one of them.
31
32 I rely on scripts to pull down the rsync metadata to expedite this
33 process. eg. rsync <host>/gentoo-portage/metadata/md5-cache/. Git has
34 no easy sub-tree download equivalent that I know of.
35
36 Brian

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies