Gentoo Archives: gentoo-portage-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [PATCH] Make manifest-required-hashes configurable
Date: Tue, 07 Nov 2017 20:25:52
Message-Id: 1510086342.14718.0.camel@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH] Make manifest-required-hashes configurable by Zac Medico
1 W dniu wto, 07.11.2017 o godzinie 11∶21 -0800, użytkownik Zac Medico
2 napisał:
3 > On 11/06/2017 10:23 AM, Michał Górny wrote:
4 > > W dniu pon, 06.11.2017 o godzinie 08∶52 -0800, użytkownik Zac Medico
5 > > napisał:
6 > > > On 11/06/2017 08:47 AM, Michał Górny wrote:
7 > > > > W dniu pon, 06.11.2017 o godzinie 08∶44 -0800, użytkownik Zac Medico
8 > > > > napisał:
9 > > > > > On 11/06/2017 07:27 AM, Michał Górny wrote:
10 > > > > > > @@ -148,15 +147,21 @@ class Manifest(object):
11 > > > > > > self.pkgdir = _unicode_decode(pkgdir).rstrip(os.sep) + os.sep
12 > > > > > > self.fhashdict = {}
13 > > > > > > self.hashes = set()
14 > > > > > > + self.required_hashes = set()
15 > > > > > >
16 > > > > > > if hashes is None:
17 > > > > > > hashes = MANIFEST2_HASH_DEFAULTS
18 > > > > > > + if required_hashes is None:
19 > > > > > > + required_hashes = hashes
20 > > > > > >
21 > > > > > > self.hashes.update(hashes)
22 > > > > > > self.hashes.difference_update(hashname for hashname in \
23 > > > > > > list(self.hashes) if hashname not in get_valid_checksum_keys())
24 > > > > > > self.hashes.add("size")
25 > > > > > > - self.hashes.add(MANIFEST2_REQUIRED_HASH)
26 > > > > > > +
27 > > > > > > + self.required_hashes.update(required_hashes)
28 > > > > > > + self.required_hashes.intersection_update(self.hashes)
29 > > > > > > +
30 > > > > >
31 > > > > > Requiring that presence of digests for all manifest-hashes can be
32 > > > > > problematic, because when we extend manifest-hashes it means that nobody
33 > > > > > can regenerate a particular manifest without fetching *all* distfiles
34 > > > > > referenced in the Manifest. It's much more friendly if we generate the
35 > > > > > missing digests when the distfiles happen to be available in DISTDIR,
36 > > > > > which is how the portage currently behaves.
37 > > > >
38 > > > > This doesn't change. The intersection only strips away hashes that are
39 > > > > not in 'self.hashes'.
40 > > >
41 > > > Then the bug must have been introduced by changes to behavior of
42 > > > digestgen in the same patch. Try adding MD5 to manifest-hashes, and then
43 > > > try generating a manifest with no internet connection and DISTDIR set to
44 > > > an empty directory. You'll see that it fails with the patch, but
45 > > > succeeds with the current master branch.
46 > >
47 > > You're suppose to add 'manifest-required-hashes' to layout.conf if you
48 > > want to introduce the partial fetching behavior.
49 >
50 > Ah, ok, that works for me. Please add a description of
51 > manifest-required-hashes after manifest\-hashes in man/portage.5.
52
53 Ah, my grep didn't catch that because of the backslash!
54
55 --
56 Best regards,
57 Michał Górny