Gentoo Archives: gentoo-portage-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [PATCH] Make manifest-required-hashes configurable
Date: Mon, 06 Nov 2017 18:23:53
Message-Id: 1509992626.1382.12.camel@gentoo.org
In Reply to: Re: [gentoo-portage-dev] [PATCH] Make manifest-required-hashes configurable by Zac Medico
1 W dniu pon, 06.11.2017 o godzinie 08∶52 -0800, użytkownik Zac Medico
2 napisał:
3 > On 11/06/2017 08:47 AM, Michał Górny wrote:
4 > > W dniu pon, 06.11.2017 o godzinie 08∶44 -0800, użytkownik Zac Medico
5 > > napisał:
6 > > > On 11/06/2017 07:27 AM, Michał Górny wrote:
7 > > > > @@ -148,15 +147,21 @@ class Manifest(object):
8 > > > > self.pkgdir = _unicode_decode(pkgdir).rstrip(os.sep) + os.sep
9 > > > > self.fhashdict = {}
10 > > > > self.hashes = set()
11 > > > > + self.required_hashes = set()
12 > > > >
13 > > > > if hashes is None:
14 > > > > hashes = MANIFEST2_HASH_DEFAULTS
15 > > > > + if required_hashes is None:
16 > > > > + required_hashes = hashes
17 > > > >
18 > > > > self.hashes.update(hashes)
19 > > > > self.hashes.difference_update(hashname for hashname in \
20 > > > > list(self.hashes) if hashname not in get_valid_checksum_keys())
21 > > > > self.hashes.add("size")
22 > > > > - self.hashes.add(MANIFEST2_REQUIRED_HASH)
23 > > > > +
24 > > > > + self.required_hashes.update(required_hashes)
25 > > > > + self.required_hashes.intersection_update(self.hashes)
26 > > > > +
27 > > >
28 > > > Requiring that presence of digests for all manifest-hashes can be
29 > > > problematic, because when we extend manifest-hashes it means that nobody
30 > > > can regenerate a particular manifest without fetching *all* distfiles
31 > > > referenced in the Manifest. It's much more friendly if we generate the
32 > > > missing digests when the distfiles happen to be available in DISTDIR,
33 > > > which is how the portage currently behaves.
34 > >
35 > > This doesn't change. The intersection only strips away hashes that are
36 > > not in 'self.hashes'.
37 >
38 > Then the bug must have been introduced by changes to behavior of
39 > digestgen in the same patch. Try adding MD5 to manifest-hashes, and then
40 > try generating a manifest with no internet connection and DISTDIR set to
41 > an empty directory. You'll see that it fails with the patch, but
42 > succeeds with the current master branch.
43
44 You're suppose to add 'manifest-required-hashes' to layout.conf if you
45 want to introduce the partial fetching behavior.
46
47 --
48 Best regards,
49 Michał Górny

Replies