Gentoo Archives: gentoo-project

From: Matthew Thode <prometheanfire@g.o>
To: gentoo-project@l.g.o
Subject: Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method
Date: Sun, 16 Dec 2018 21:10:22
Message-Id: 20181216211007.3if7u53d5tpcetap@gentoo.org
In Reply to: Re: [gentoo-project] RFC: Dropping rsync as a tree distribution method by Rich Freeman
1 On 18-12-16 06:34:07, Rich Freeman wrote:
2 > On Sat, Dec 15, 2018 at 11:15 PM Alec Warner <antarus@g.o> wrote:
3 > >
4 > > [1] Rich talked about some downsides earlier at https://lwn.net/Articles/759539/; but while these are challenges (some fixable) they are not necessarily blockers.
5 >
6 > The thread has already touched on a few of those comments. Despite
7 > only six months elapsing since I wrote that email, #1 no longer
8 > applies, and it sounds like #4 may not be as much of a concern. As
9 > you've already stated #3 can be easily addressed - setting up a git
10 > mirror is very easy.
11 >
12 > I think #2 is more of a fundamental design difference that probably
13 > will never go away. If your tree is a year old then git WILL take
14 > longer and transfer more data than rsync. My guess is that it will
15 > also cost more IO server-side than rsync, but it probably will be
16 > cheaper in CPU. However, I bet that 95% of our users sync weekly or
17 > daily and in that use case it is going to go a lot faster, and
18 > probably be less mirror load as well, and it will be a TON less IO
19 > load on the client side. I'm not sure how much IO cost there is to
20 > git garbage collection - that might offset this in the common shallow
21 > clone scenario.
22 >
23 > I'd suggest that those with concerns give it a shot using Zac's
24 > suggested settings and see how it goes. Really all you have to do is
25 > delete your local repo and adjust your sync settings and resync. I
26 > think the local disk use is going to be the biggest source of user
27 > objection and I'm interested in what people observe here.
28 >
29
30 I wonder if we can add a little logic to help at least a little bit on
31 the yearly syncers. If over a 6 months, remove old git sync'd dir and
32 replance with new shallow clone? Not perfect, but workable maybe.
33
34 Do we need to tell users to set up a git gc cron job or does portage
35 handle that for us now?
36
37 --
38 Matthew Thode (prometheanfire)

Attachments

File name MIME type
signature.asc application/pgp-signature