Gentoo Archives: gentoo-commits

From: "Robin H. Johnson (robbat2)" <robbat2@g.o>
To: gentoo-commits@l.g.o
Subject: [gentoo-commits] gentoo commit in xml/htdocs/proj/en/glep: glep-0059.html glep-0059.txt
Date: Sun, 31 Jan 2010 07:55:49
Message-Id: E1NbUeo-0008N7-Dx@stork.gentoo.org
1 robbat2 10/01/31 07:55:46
2
3 Modified: glep-0059.html glep-0059.txt
4 Log:
5 Revise GLEP59 per Calchan questions: Python 2.5 is widely deployed now and provides SHA512. RIPEMD160 is broken. WHIRLPOOL added. Migration plan detail added.
6
7 Revision Changes Path
8 1.5 xml/htdocs/proj/en/glep/glep-0059.html
9
10 file : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?rev=1.5&view=markup
11 plain: http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?rev=1.5&content-type=text/plain
12 diff : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?r1=1.4&r2=1.5
13
14 Index: glep-0059.html
15 ===================================================================
16 RCS file: /var/cvsroot/gentoo/xml/htdocs/proj/en/glep/glep-0059.html,v
17 retrieving revision 1.4
18 retrieving revision 1.5
19 diff -p -w -b -B -u -u -r1.4 -r1.5
20 --- glep-0059.html 13 Jan 2010 03:28:33 -0000 1.4
21 +++ glep-0059.html 31 Jan 2010 07:55:45 -0000 1.5
22 @@ -47,7 +47,7 @@
23 </tr>
24 <tr class="field"><th class="field-name">Updates:</th><td class="field-body">44</td>
25 </tr>
26 -<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">December 2009</td>
27 +<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">December 2009, January 2010</td>
28 </tr>
29 </tbody>
30 </table>
31 @@ -61,10 +61,8 @@
32 <li><a class="reference internal" href="#the-bad-news" id="id4">The bad news</a></li>
33 <li><a class="reference internal" href="#how-fast-can-md5-be-broken" id="id5">How fast can MD5 be broken?</a></li>
34 <li><a class="reference internal" href="#the-good-news" id="id6">The good news</a></li>
35 -<li><a class="reference internal" href="#what-should-be-done" id="id7">What should be done</a><ul>
36 -<li><a class="reference internal" href="#checksum-depreciation" id="id8">Checksum depreciation</a></li>
37 -</ul>
38 -</li>
39 +<li><a class="reference internal" href="#what-should-be-done" id="id7">What should be done</a></li>
40 +<li><a class="reference internal" href="#checksum-depreciation-timing" id="id8">Checksum depreciation timing</a></li>
41 </ul>
42 </li>
43 <li><a class="reference internal" href="#backwards-compatibility" id="id9">Backwards Compatibility</a></li>
44 @@ -100,16 +98,19 @@ is that multiple checksums would be an i
45 could not provably quantify the amount of security this added.
46 The really bad news, is that this position is completely and utterly
47 wrong. Many of you will be aghast at this. There is extremely little
48 -added security in multiple checksums [J04]. For any set of checksums,
49 -the actual strength lies in that of the strongest checksum.</p>
50 +added security in multiple checksums as noted by Joux [J04]. For any set
51 +of checksums, the actual strength lies in that of the strongest
52 +checksum.</p>
53 +<p>Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5,
54 +HAVAL-128 and RIPEMD families of hashes.</p>
55 </div>
56 <div class="section" id="how-fast-can-md5-be-broken">
57 <h2><a class="toc-backref" href="#id5">How fast can MD5 be broken?</a></h2>
58 -<p>For a general collision, not a pre-image attack, since the original
59 -announcement by Wang et al [W04], the time required to break MD5 has
60 -been massively reduced. Originally at 1 hour on a near-supercomputer
61 -(IBM P690) and estimated at 64 hours with a Pentium-3 1.7Ghz. This has
62 -gone down to less than in two years, to 17 seconds [K06a]!</p>
63 +<p>For a general collision, not a pre-image attack, since the announcement
64 +by Wang et al [W04], the time required to break MD5 has been massively
65 +reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and
66 +estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to
67 +less than in two years, to 17 seconds [K06a].</p>
68 <p>08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours
69 03/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours
70 11/2005 - 5 hours, Pentium-4 1.7Ghz = 8.5 Ghz-Hours
71 @@ -120,26 +121,24 @@ may be broken over the course of 2 years
72 &gt;2000x), then existing checksums do not stand a significant chance of
73 survival in the future. We should thus accept that whatever checksums we
74 are using today, will be broken in the near future, and plan as best as
75 -possible. (A brief review [H04] of the present SHA1 attacks indicates an
76 +possible. (A brief review [H04] of the SHA1 attacks indicates an
77 improvement of ~600x in the same timespan).</p>
78 <p>And for those that claim implementation of these procedures is not yet
79 feasible, see [K06b] for an application that can produce two
80 -self-extracting .exe files, with identical MD5s, and whatever payload
81 -you want.</p>
82 +self-extracting EXE files, with identical MD5s, and whatever payload you
83 +want.</p>
84 </div>
85 <div class="section" id="the-good-news">
86 <h2><a class="toc-backref" href="#id6">The good news</a></h2>
87 -<p>Of the checksums presently used by Manifest2, one stands close to being
88 -completely broken: SHA1. The SHA2 series has suffered some attacks, but
89 -still remains reasonably solid [G07],[K08]. No attacks against RIPEMD160
90 -have been published, however it is constructed in the same manner as
91 -MD5, SHA1 and SHA2, so is also vulnerable to the new methods of
92 -cryptanalysis [H04].</p>
93 +<p>Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160),
94 +one stands close to being completely broken: SHA1; and another is
95 +significantly weakened: RIPEMD160. The SHA2 series has suffered some
96 +attacks, but still remains reasonably solid [G07],[K08].</p>
97 <p>To reduce the potential for future problems and any single checksum
98 break leading to a rapid decrease in security, we should incorporate the
99 strongest hash available from each family of checksums, and be prepared
100 to retire old checksums actively, unless there is a overriding reason to
101 -keep a specific checksum.</p>
102 +keep a specific checksum, such as part of a migration plan.</p>
103 </div>
104 <div class="section" id="what-should-be-done">
105 <h2><a class="toc-backref" href="#id7">What should be done</a></h2>
106 @@ -150,23 +149,30 @@ from Manifest2 files, once all old Porta
107 sufficient time to upgrade. We should be prepared to add stronger
108 checksums wherever possible, and to remove those that have been
109 defeated.</p>
110 -<p>An unsupported hash is not considered to be a failure unless no
111 -supported hashes are available.</p>
112 -<div class="section" id="checksum-depreciation">
113 -<h3><a class="toc-backref" href="#id8">Checksum depreciation</a></h3>
114 -<p>For the current Portage, SHA1 should be gradually removed, as presents
115 -no advantages over SHA256. Beyond one specific problem (see the next
116 -paragraph), we should add SHA512 (SHA2, 512 bit size), the Whirlpool
117 -checksum (standardized checksum, with no known weaknesses). In future,
118 -as stream-based checksums are developed (in response to the development
119 -by NIST [AHS]), they should be considered and used.</p>
120 -<p>There is one temporary stumbling block at hand - the existing Portage
121 -infrastructure does not support SHA384/512 or Whirlpool, thus hampering
122 -their immediate acceptance. SHA512 is available in Python 2.5, while
123 -SHA1 is already available in Python 2.4. After Python2.5 is established
124 -in a Gentoo media release, that would be a suitable time to remove SHA1
125 -from Manifest2 files.</p>
126 -</div>
127 +<p>As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms.
128 +In future, as stream-based checksums are developed (in response to the
129 +development by NIST [AHS]), they should be considered and used.</p>
130 +<p>The SHA512 algorithm is available in Python 2.5, which has been a
131 +dependency of Portage since approximately Python 2.1.6.13.</p>
132 +<p>The WHIRLPOOL checksum is not available within the PyCrypto library or
133 +hashlib that is part of Python 2.5, but there are multiple alternative
134 +Python implementations available, ranging from pure Python to C-based
135 +(python-mhash).</p>
136 +<p>The existence unsupported hash is not considered to be a failure unless
137 +no supported hashes are available for a given Manifest entry.</p>
138 +</div>
139 +<div class="section" id="checksum-depreciation-timing">
140 +<h2><a class="toc-backref" href="#id8">Checksum depreciation timing</a></h2>
141 +<p>For the current Portage, both SHA1 and RIPEMD160 should be immediately
142 +removed, as they present no advantages over the already present SHA256.
143 +SHA256 cannot be replaced immediately with SHA512, as existing Portage
144 +versions need at least one supported algorithm present (SHA256 support
145 +was added in June 2006), so it must be retained for some while.</p>
146 +<p>Immediately:
147 +- Add WHIRLPOOL and SHA512.
148 +- Remove SHA1 and RIPEMD160.</p>
149 +<p>After the majority of Portage installations include SHA512 support:
150 +- Remove SHA256.</p>
151 </div>
152 </div>
153 <div class="section" id="backwards-compatibility">
154 @@ -238,7 +244,7 @@ Open Publication License, v1.0.</p>
155 <div class="footer">
156 <hr class="footer" />
157 <a class="reference external" href="glep-0059.txt">View document source</a>.
158 -Generated on: 2010-01-13 03:27 UTC.
159 +Generated on: 2010-01-31 07:53 UTC.
160 Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
161
162 </div>
163
164
165
166 1.5 xml/htdocs/proj/en/glep/glep-0059.txt
167
168 file : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?rev=1.5&view=markup
169 plain: http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?rev=1.5&content-type=text/plain
170 diff : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?r1=1.4&r2=1.5
171
172 Index: glep-0059.txt
173 ===================================================================
174 RCS file: /var/cvsroot/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt,v
175 retrieving revision 1.4
176 retrieving revision 1.5
177 diff -p -w -b -B -u -u -r1.4 -r1.5
178 --- glep-0059.txt 13 Jan 2010 03:26:53 -0000 1.4
179 +++ glep-0059.txt 31 Jan 2010 07:55:45 -0000 1.5
180 @@ -1,7 +1,7 @@
181 GLEP: 59
182 Title: Manifest2 hash policies and security implications
183 -Version: $Revision: 1.4 $
184 -Last-Modified: $Date: 2010/01/13 03:26:53 $
185 +Version: $Revision: 1.5 $
186 +Last-Modified: $Date: 2010/01/31 07:55:45 $
187 Author: Robin Hugh Johnson <robbat2@g.o>,
188 Status: Draft
189 Type: Standards Track
190 @@ -10,7 +10,7 @@ Requires: 44
191 Created: October 2006
192 Updated: November 2007, June 2008, July 2008, October 2008, January 2010
193 Updates: 44
194 -Post-History: December 2009
195 +Post-History: December 2009, January 2010
196
197 Abstract
198 ========
199 @@ -39,16 +39,20 @@ is that multiple checksums would be an i
200 could not provably quantify the amount of security this added.
201 The really bad news, is that this position is completely and utterly
202 wrong. Many of you will be aghast at this. There is extremely little
203 -added security in multiple checksums [J04]. For any set of checksums,
204 -the actual strength lies in that of the strongest checksum.
205 +added security in multiple checksums as noted by Joux [J04]. For any set
206 +of checksums, the actual strength lies in that of the strongest
207 +checksum.
208 +
209 +Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5,
210 +HAVAL-128 and RIPEMD families of hashes.
211
212 How fast can MD5 be broken?
213 ---------------------------
214 -For a general collision, not a pre-image attack, since the original
215 -announcement by Wang et al [W04], the time required to break MD5 has
216 -been massively reduced. Originally at 1 hour on a near-supercomputer
217 -(IBM P690) and estimated at 64 hours with a Pentium-3 1.7Ghz. This has
218 -gone down to less than in two years, to 17 seconds [K06a]!
219 +For a general collision, not a pre-image attack, since the announcement
220 +by Wang et al [W04], the time required to break MD5 has been massively
221 +reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and
222 +estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to
223 +less than in two years, to 17 seconds [K06a].
224
225 08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours
226 03/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours
227 @@ -61,28 +65,26 @@ may be broken over the course of 2 years
228 >2000x), then existing checksums do not stand a significant chance of
229 survival in the future. We should thus accept that whatever checksums we
230 are using today, will be broken in the near future, and plan as best as
231 -possible. (A brief review [H04] of the present SHA1 attacks indicates an
232 +possible. (A brief review [H04] of the SHA1 attacks indicates an
233 improvement of ~600x in the same timespan).
234
235 And for those that claim implementation of these procedures is not yet
236 feasible, see [K06b] for an application that can produce two
237 -self-extracting .exe files, with identical MD5s, and whatever payload
238 -you want.
239 +self-extracting EXE files, with identical MD5s, and whatever payload you
240 +want.
241
242 The good news
243 -------------
244 -Of the checksums presently used by Manifest2, one stands close to being
245 -completely broken: SHA1. The SHA2 series has suffered some attacks, but
246 -still remains reasonably solid [G07],[K08]. No attacks against RIPEMD160
247 -have been published, however it is constructed in the same manner as
248 -MD5, SHA1 and SHA2, so is also vulnerable to the new methods of
249 -cryptanalysis [H04].
250 +Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160),
251 +one stands close to being completely broken: SHA1; and another is
252 +significantly weakened: RIPEMD160. The SHA2 series has suffered some
253 +attacks, but still remains reasonably solid [G07],[K08].
254
255 To reduce the potential for future problems and any single checksum
256 break leading to a rapid decrease in security, we should incorporate the
257 strongest hash available from each family of checksums, and be prepared
258 to retire old checksums actively, unless there is a overriding reason to
259 -keep a specific checksum.
260 +keep a specific checksum, such as part of a migration plan.
261
262 What should be done
263 -------------------
264 @@ -94,24 +96,35 @@ sufficient time to upgrade. We should be
265 checksums wherever possible, and to remove those that have been
266 defeated.
267
268 -An unsupported hash is not considered to be a failure unless no
269 -supported hashes are available.
270 +As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms.
271 +In future, as stream-based checksums are developed (in response to the
272 +development by NIST [AHS]), they should be considered and used.
273 +
274 +The SHA512 algorithm is available in Python 2.5, which has been a
275 +dependency of Portage since approximately Python 2.1.6.13.
276 +
277 +The WHIRLPOOL checksum is not available within the PyCrypto library or
278 +hashlib that is part of Python 2.5, but there are multiple alternative
279 +Python implementations available, ranging from pure Python to C-based
280 +(python-mhash).
281 +
282 +The existence unsupported hash is not considered to be a failure unless
283 +no supported hashes are available for a given Manifest entry.
284 +
285 +Checksum depreciation timing
286 +----------------------------
287 +For the current Portage, both SHA1 and RIPEMD160 should be immediately
288 +removed, as they present no advantages over the already present SHA256.
289 +SHA256 cannot be replaced immediately with SHA512, as existing Portage
290 +versions need at least one supported algorithm present (SHA256 support
291 +was added in June 2006), so it must be retained for some while.
292 +
293 +Immediately:
294 +- Add WHIRLPOOL and SHA512.
295 +- Remove SHA1 and RIPEMD160.
296
297 -Checksum depreciation
298 -~~~~~~~~~~~~~~~~~~~~~
299 -For the current Portage, SHA1 should be gradually removed, as presents
300 -no advantages over SHA256. Beyond one specific problem (see the next
301 -paragraph), we should add SHA512 (SHA2, 512 bit size), the Whirlpool
302 -checksum (standardized checksum, with no known weaknesses). In future,
303 -as stream-based checksums are developed (in response to the development
304 -by NIST [AHS]), they should be considered and used.
305 -
306 -There is one temporary stumbling block at hand - the existing Portage
307 -infrastructure does not support SHA384/512 or Whirlpool, thus hampering
308 -their immediate acceptance. SHA512 is available in Python 2.5, while
309 -SHA1 is already available in Python 2.4. After Python2.5 is established
310 -in a Gentoo media release, that would be a suitable time to remove SHA1
311 -from Manifest2 files.
312 +After the majority of Portage installations include SHA512 support:
313 +- Remove SHA256.
314
315 Backwards Compatibility
316 =======================