Gentoo Archives: gentoo-portage-dev

From: Zac Medico <zmedico@g.o>
To: gentoo-portage-dev@l.g.o
Cc: Zac Medico <zmedico@g.o>
Subject: [gentoo-portage-dev] [PATCH] rsync: default to sync-rsync-verify-jobs = 1 (bug 650696)
Date: Fri, 13 Apr 2018 17:10:39
Message-Id: 20180413171017.4591-1-zmedico@gentoo.org
1 Some users have reported that using all processors to verify
2 manifests results in overloading, therefore default to using
3 a single processor. On modern hardware, verification of the
4 gentoo repository completes in less than 20 seconds, so using
5 multiple processors is not really necessary. Also, gemato-13.0
6 disables parallel verification due to the unresolved deadlock
7 issue reported in bug 647964, so this brings the default
8 portage configuration into alignment with current gemato
9 behavior.
10
11 Bug: https://bugs.gentoo.org/650696
12 Bug: https://bugs.gentoo.org/647964
13 ---
14 cnf/repos.conf | 1 +
15 man/portage.5 | 8 +++++---
16 pym/portage/sync/modules/rsync/rsync.py | 7 ++++++-
17 3 files changed, 12 insertions(+), 4 deletions(-)
18
19 diff --git a/cnf/repos.conf b/cnf/repos.conf
20 index 5759b8b43..987be6462 100644
21 --- a/cnf/repos.conf
22 +++ b/cnf/repos.conf
23 @@ -6,6 +6,7 @@ location = /usr/portage
24 sync-type = rsync
25 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
26 auto-sync = yes
27 +sync-rsync-verify-jobs = 1
28 sync-rsync-verify-metamanifest = yes
29 sync-rsync-verify-max-age = 24
30 sync-openpgp-key-path = /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
31 diff --git a/man/portage.5 b/man/portage.5
32 index 2c3a75ccd..6e4089ae5 100644
33 --- a/man/portage.5
34 +++ b/man/portage.5
35 @@ -1105,9 +1105,11 @@ Ignore vcs directories that may be present in the repository. It is the
36 user's responsibility to set sync-rsync-extra-opts to protect vcs
37 directories if appropriate.
38 .TP
39 -.B sync\-rsync\-verify\-jobs
40 -Number of parallel jobs to use when verifying nested Manifests. Defaults
41 -to the apparent number of processors.
42 +.B sync\-rsync\-verify\-jobs = 1
43 +Number of parallel jobs to use when verifying nested Manifests. When
44 +set to 0, this will use the apparent number of processors if parallel
45 +verification is supported by the installed version of app-portage/gemato.
46 +Defaults to 1.
47 .TP
48 .B sync\-rsync\-verify\-max\-age
49 Warn if repository is older than the specified number of days. Disabled
50 diff --git a/pym/portage/sync/modules/rsync/rsync.py b/pym/portage/sync/modules/rsync/rsync.py
51 index 763f41699..de8327a55 100644
52 --- a/pym/portage/sync/modules/rsync/rsync.py
53 +++ b/pym/portage/sync/modules/rsync/rsync.py
54 @@ -107,12 +107,17 @@ class RsyncSync(NewBase):
55 if self.verify_jobs is not None:
56 try:
57 self.verify_jobs = int(self.verify_jobs)
58 - if self.verify_jobs <= 0:
59 + if self.verify_jobs < 0:
60 raise ValueError(self.verify_jobs)
61 except ValueError:
62 writemsg_level("!!! sync-rsync-verify-jobs not a positive integer: %s\n" % (self.verify_jobs,),
63 level=logging.WARNING, noiselevel=-1)
64 self.verify_jobs = None
65 + else:
66 + if self.verify_jobs == 0:
67 + # Use the apparent number of processors if gemato
68 + # supports it.
69 + self.verify_jobs = None
70 # Support overriding max age.
71 self.max_age = self.repo.module_specific_options.get(
72 'sync-rsync-verify-max-age', '')
73 --
74 2.13.6