Gentoo Archives: gentoo-portage-dev

From: Zac Medico <zmedico@g.o>
To: "Michał Górny" <mgorny@g.o>
Cc: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)
Date: Sat, 05 Nov 2016 22:56:26
Message-Id: 845bf71c-87c9-2745-9f75-8d537ee89782@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008) by "Michał Górny"
1 On 11/05/2016 03:22 PM, Michał Górny wrote:
2 > On Sat, 5 Nov 2016 15:11:10 -0700
3 > Zac Medico <zmedico@g.o> wrote:
4 >
5 >> On 11/05/2016 02:50 PM, Michał Górny wrote:
6 >>> On Sat, 5 Nov 2016 13:43:15 -0700
7 >>> Zac Medico <zmedico@g.o> wrote:
8 >>>
9 >>>> This is necessary in order to avoid "There are too many unreachable
10 >>>> loose objects" warnings from automatic git gc calls.
11 >>>>
12 >>>> X-Gentoo-Bug: 599008
13 >>>> X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=599008
14 >>>> ---
15 >>>> pym/portage/sync/modules/git/git.py | 6 ++++++
16 >>>> 1 file changed, 6 insertions(+)
17 >>>>
18 >>>> diff --git a/pym/portage/sync/modules/git/git.py b/pym/portage/sync/modules/git/git.py
19 >>>> index f288733..c90cf88 100644
20 >>>> --- a/pym/portage/sync/modules/git/git.py
21 >>>> +++ b/pym/portage/sync/modules/git/git.py
22 >>>> @@ -101,6 +101,12 @@ class GitSync(NewBase):
23 >>>> writemsg_level(msg + "\n", level=logging.ERROR, noiselevel=-1)
24 >>>> return (e.returncode, False)
25 >>>>
26 >>>> + # For shallow fetch, unreachable objects must be pruned
27 >>>> + # manually, since otherwise automatic git gc calls will
28 >>>> + # eventually warn about them (see bug 599008).
29 >>>> + subprocess.call(['git', 'prune'],
30 >>>> + cwd=portage._unicode_encode(self.repo.location))
31 >>>> +
32 >>>> git_cmd_opts += " --depth %d" % self.repo.sync_depth
33 >>>> git_cmd = "%s fetch %s%s" % (self.bin_command,
34 >>>> remote_branch.partition('/')[0], git_cmd_opts)
35 >>>
36 >>> Does it have a performance impact?
37 >>
38 >> Yes, it takes about 20 seconds on my laptop. I suppose we could make
39 >> this an optional thing, so that those people can do it manually if they
40 >> want.
41 >
42 > So we have improvement from at most few seconds for normal 'git pull'
43 > to around a minute for shallow pull?
44
45 Well we've got a least 3 resources to consider:
46
47 1) network bandwidth
48 2) disk usage
49 3) sync time
50
51 For me, sync time doesn't really matter that much, but I suppose it
52 might for some people.
53 --
54 Thanks,
55 Zac

Replies