Gentoo Archives: gentoo-portage-dev

From: "Robin H. Johnson" <robbat2@g.o>
To: gentoo-portage-dev@l.g.o
Subject: [gentoo-portage-dev] Manifest signing
Date: Sat, 19 Nov 2005 06:03:13
1 Hi,
3 After my post to -core about how to move ahead with signing, I thought
4 the next best place to continue is in a discussion of how Portage
5 handles manifests and their signatures.
7 First, the blatantly obvious, for the benefit of same developers, even
8 though it's not relevant to signing. It is still a weak-point and does
9 need to be addressed. Multiple-hashes!
10 Ok, so Wang et al. showed you can break MD5 about an hour, on modern
11 machines, in a specific fashion. Their paper also discussed the effects
12 of their new mathematical work had on SHA1, which was making it
13 significantly easier to break. However, the nature of the math is that
14 it's still too computationally intensive for anybody but NSA-type
15 attackers to defeat both SHA1 and MD5 at the same time.
16 Simple solution - just put both hashes into the Manifest/digest.
18 Now onto the real stuff. The existing Manifest system has a very
19 specific problem in ensuring the trustworthy-ness of files.
20 It's quickest to explain via example.
21 1. cat-foo/bar has a complete, signed Manifest, and all of the content
22 is verified.
23 2. Dev A commits a change to the package, the Manifest is regenerated,
24 and Dev A doesn't sign it.
25 3. Dev B goes commits something to the package, maybe not even to the
26 same file. However Dev B does
27 sign the newly generated Manifest.
28 The change made by Dev A was not certified via Dev A's signature,
29 however when Dev B came to change it, it suddenly became
30 trusted/certified code. Now what if Dev A was an attacker with a stolen
31 SSH key? And worse, Dev B didn't actually check the ebuild, as he was
32 using ekeyword to stable a lot of packages? (ignoring the fact that
33 many-eyes should have looked at the ebuild, but really, how long will
34 it take before the change it actually noticed, esp. if there is no
35 accompanying version bump).
37 Everybody on board with the problem?
39 Now a moments discussion about a solution.
40 One of my goals in examining signing is to make true end-to-end
41 verification via checksums+signing possible - to avoid any and all forms
42 of injection attacks going unnoticed.
44 Once the entire tree is signed, then the process of development will be
45 altered towards the following:
46 1) [dev] cvs up + verify all incoming changes are signed (if paranoid)
47 2) [dev] work on something, repoman ci with initial Manifest.
48 3) [dev] signed manifest commited.
49 4) [cvs server] queue newly signed manifest for verification
50 5) [cvs server - async] verify all newly signed manifests, if one
51 fails, block CVS -> rsync update to allow manual fix/revert. (Yes, this
52 is policing).
53 7) [user] emerge --sync
54 8) [user] emerge foobar - and everything is verified as signed
56 Solving the problem isn't hard. All we need is to consider each change
57 to a Manifest separately.
59 Ergo, instead of a Manifest being re-generated each time, it needs to
60 act like a FIFO queue.
61 Each queue element consists of:
62 - checksum/existing Manifest element of items changed in that action ONLY (however it should be possible to forcibly include files).
63 - Signature around the above checksum.
65 So now the new Manifest structure looks roughly like this (abbreviated):
66 -- PGP
67 MD5 ...
68 MD5 ...
69 -- SIG
70 -- SIG
71 -- PGP
72 MD5 ...
73 -- SIG
74 -- SIG
75 etc.
77 This has one important implication for backwards compatibility in
78 checking of Manifests.
79 In the case that a filename appears more than once in the file, only
80 the last instance of it should be used, as that is the one that relates
81 to the current version of the file. It's 4 lines of code in the current
82 portage that need to be removed for this to work (see my -core post for
83 where exactly).
85 Generation of the above is reasonably simple, just make the checksums in
86 a string, clear-sign via gpg separately/tmpfile, append to Manifest.
88 There is one last part that does need to be taken care of.
89 At what point is it safe to remove a checksum/signature from the
90 Manifest?
91 You cannot remove a single checksum from inside a signature block, as
92 that invalidates the signature.
93 So instead, now we need to wait for the existence more recent
94 checksums/signatures to exist for every item in a block before the old
95 block can be removed
97 This comes into play again as us needing to be able to force recreation
98 of checksums/signatures - as the key that did the original signature
99 might no longer be valid (for various reasons). So repoman needs to
100 have some option to do:
101 repoman forcesign $FILES
103 Notes:
104 There were some concerns about the speed of Manifest checking.
105 I did some simple benchmarking, using all existing signed Manifests
106 (40% of the tree). Using a Pentium-D 3Ghz, running on only one core, it
107 took 43 seconds to check all 4000 Manifests. 90% of the time was in the
108 setup/teardown for gpg.
110 Getting around the setup/teardown time problem basically means we need
111 something of our own to interface with the gpg API (gpgme). This is not
112 too hard, GPG ships with an example that is most of the way there
113 already - 'gpgv'.
115 I'm comfortable with the GPG/GPGME codebase, and writing crypto-related
116 code, so I'm going to tackle that, starting after I've eaten dinner
117 tonight.
119 Even still, re-checking every digest in the tree should not happen on
120 every CVS->rsync window. It's computationally pointless. Just check the
121 changes.
123 --
124 Robin Hugh Johnson
125 E-Mail : robbat2@g.o
126 GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85


Subject Author
Re: [gentoo-portage-dev] Manifest signing Jason Stubbs <jstubbs@g.o>
Re: [gentoo-portage-dev] Manifest signing Ciaran McCreesh <ciaranm@g.o>
Re: [gentoo-portage-dev] Manifest signing Brian Harring <ferringb@g.o>
Re: [gentoo-portage-dev] Manifest signing Marius Mauch <genone@g.o>