Gentoo Archives: gentoo-dev

From: "Robin H. Johnson" <robbat2@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] proposal: use only one hash function in manifest files
Date: Wed, 06 Apr 2022 17:23:35
Message-Id: robbat2-20220406T165003-021185447Z@orbis-terrarum.net
In Reply to: Re: [gentoo-dev] proposal: use only one hash function in manifest files by "Jason A. Donenfeld"
1 On Wed, Apr 06, 2022 at 02:15:02AM +0200, Jason A. Donenfeld wrote:
2 > 2) Comparability: other distros use SHA2-512, as well as various
3 > upstreams, which means we can compare our hashes to theirs easily.
4 Can we expand on this specific thread for a moment?
5
6 I was the author of GLEP59 about changing the Manifest hashes, and I
7 noted at the time, with references, that the effective strength of a set
8 of hashes is only that of the strongest hash.
9
10 One of my regrets from GLEP59 is that it's made it harder for use cases
11 outside of the normal user distfile workflow.
12
13 The use case that impacted me the most was being able to compare our
14 distfiles were over time vs external sources, esp. if the file goes
15 missing or was fetch-restricted and we can't produce a new hash of it.
16 Maybe upstream only ever published SHA1/SHA256, and we only ever
17 calculated SHA512/BLAKE2b on the file. Since we never had hashes from
18 both sides at the same time, we cannot prove it was the same file.
19
20 We need to be able to ship one or more hashes to users, for the specific
21 use case of validating the distfiles they download.
22
23 As a developer, I'd like to be able to track the other hashes for a
24 file, without forcing ourselves to retain the file. This might be to
25 compare with upstream published hashes, or to compare with other
26 distros.
27
28 In fact it would be really nice to have a semi-automated pipeline to
29 plug in signed upstream hashes to our Manifests, and make it possibly to
30 prove our new SHA512/BLAKE2B hash was taken over the correct input in
31 the first place, and there wasn't any subtle supply-chain attack early
32 in the packaging process.
33
34 Where would those hashes go? They don't need to be in the Manifest, or
35 at the very least they don't need to be distributed via rsync to users
36 (it only costs a small amount of bytes to do so).
37
38 Where else could they go?
39 - Commit messages could work.
40 - Git notes to a lesser degree.
41 - alternate repos?
42
43 > A reason why some people might prefer BLAKE2b over SHA2-512 is a
44 > performance improvement. However, seeing as right now we're opening
45 > the file, reading it, computing BLAKE2b, closing the file, opening the
46 > file again, reading it again, computing SHA2-512, closing the file, I
47 > don't think performance is actually something people care about. Seen
48 > differently, removing either one of them will already give us a
49 > performance "boost" or sorts.
50 Or just only verifying the "strongest" hash gives you that boost.
51
52 I do want to check into the code that you pointed out, because I'm
53 really sure much older versions of Portage did the CORRECT thing of only
54 reading the file in a single pass.
55
56 --
57 Robin Hugh Johnson
58 Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
59 E-Mail : robbat2@g.o
60 GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
61 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies