Gentoo Archives: gentoo-project

From: Kent Fredric <kentnl@g.o>
To: gentoo-project@l.g.o
Subject: Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method
Date: Thu, 20 Dec 2018 01:19:11
Message-Id: 20181220141829.47cd9cc3@katipo2.lan
In Reply to: Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method by Georgy Yakovlev
1 On Mon, 17 Dec 2018 19:57:21 -0800
2 Georgy Yakovlev <gyakovlev@g.o> wrote:
3
4 > I'm guessing, but probably infra server is not supposed to handle load from
5 > all the users and will temporarily ban if one tries to sync more than several
6 > times per day (like rsync master does). But don't quote me on that, better ask
7 > infra.
8
9 I'd imagine the server requirements with regard to load, is less for
10 git than it is for rsync.
11
12 Partly, because I believe rsync's require tree traversal, and dynamic
13 checksumming of data on the server side for each sync.
14
15 Whereas with Git, that checksumming and traversal are essentially
16 precomputed, and the backing store can be efficiently condensed to a
17 single file, with much more efficient IO.
18
19 That is, instead of iterating through 9k+ inodes, it just opens the
20 one and chases the parent SHA1 chains.
21
22 Then your restrictions seem to amount to total bandwidth available,
23 with a little CPU and IO overhead, as opposed to a larger bandwith, CPU
24 and IO requirement.