1 |
robbat2 10/01/31 07:55:46 |
2 |
|
3 |
Modified: glep-0059.html glep-0059.txt |
4 |
Log: |
5 |
Revise GLEP59 per Calchan questions: Python 2.5 is widely deployed now and provides SHA512. RIPEMD160 is broken. WHIRLPOOL added. Migration plan detail added. |
6 |
|
7 |
Revision Changes Path |
8 |
1.5 xml/htdocs/proj/en/glep/glep-0059.html |
9 |
|
10 |
file : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?rev=1.5&view=markup |
11 |
plain: http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?rev=1.5&content-type=text/plain |
12 |
diff : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.html?r1=1.4&r2=1.5 |
13 |
|
14 |
Index: glep-0059.html |
15 |
=================================================================== |
16 |
RCS file: /var/cvsroot/gentoo/xml/htdocs/proj/en/glep/glep-0059.html,v |
17 |
retrieving revision 1.4 |
18 |
retrieving revision 1.5 |
19 |
diff -p -w -b -B -u -u -r1.4 -r1.5 |
20 |
--- glep-0059.html 13 Jan 2010 03:28:33 -0000 1.4 |
21 |
+++ glep-0059.html 31 Jan 2010 07:55:45 -0000 1.5 |
22 |
@@ -47,7 +47,7 @@ |
23 |
</tr> |
24 |
<tr class="field"><th class="field-name">Updates:</th><td class="field-body">44</td> |
25 |
</tr> |
26 |
-<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">December 2009</td> |
27 |
+<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">December 2009, January 2010</td> |
28 |
</tr> |
29 |
</tbody> |
30 |
</table> |
31 |
@@ -61,10 +61,8 @@ |
32 |
<li><a class="reference internal" href="#the-bad-news" id="id4">The bad news</a></li> |
33 |
<li><a class="reference internal" href="#how-fast-can-md5-be-broken" id="id5">How fast can MD5 be broken?</a></li> |
34 |
<li><a class="reference internal" href="#the-good-news" id="id6">The good news</a></li> |
35 |
-<li><a class="reference internal" href="#what-should-be-done" id="id7">What should be done</a><ul> |
36 |
-<li><a class="reference internal" href="#checksum-depreciation" id="id8">Checksum depreciation</a></li> |
37 |
-</ul> |
38 |
-</li> |
39 |
+<li><a class="reference internal" href="#what-should-be-done" id="id7">What should be done</a></li> |
40 |
+<li><a class="reference internal" href="#checksum-depreciation-timing" id="id8">Checksum depreciation timing</a></li> |
41 |
</ul> |
42 |
</li> |
43 |
<li><a class="reference internal" href="#backwards-compatibility" id="id9">Backwards Compatibility</a></li> |
44 |
@@ -100,16 +98,19 @@ is that multiple checksums would be an i |
45 |
could not provably quantify the amount of security this added. |
46 |
The really bad news, is that this position is completely and utterly |
47 |
wrong. Many of you will be aghast at this. There is extremely little |
48 |
-added security in multiple checksums [J04]. For any set of checksums, |
49 |
-the actual strength lies in that of the strongest checksum.</p> |
50 |
+added security in multiple checksums as noted by Joux [J04]. For any set |
51 |
+of checksums, the actual strength lies in that of the strongest |
52 |
+checksum.</p> |
53 |
+<p>Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5, |
54 |
+HAVAL-128 and RIPEMD families of hashes.</p> |
55 |
</div> |
56 |
<div class="section" id="how-fast-can-md5-be-broken"> |
57 |
<h2><a class="toc-backref" href="#id5">How fast can MD5 be broken?</a></h2> |
58 |
-<p>For a general collision, not a pre-image attack, since the original |
59 |
-announcement by Wang et al [W04], the time required to break MD5 has |
60 |
-been massively reduced. Originally at 1 hour on a near-supercomputer |
61 |
-(IBM P690) and estimated at 64 hours with a Pentium-3 1.7Ghz. This has |
62 |
-gone down to less than in two years, to 17 seconds [K06a]!</p> |
63 |
+<p>For a general collision, not a pre-image attack, since the announcement |
64 |
+by Wang et al [W04], the time required to break MD5 has been massively |
65 |
+reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and |
66 |
+estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to |
67 |
+less than in two years, to 17 seconds [K06a].</p> |
68 |
<p>08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours |
69 |
03/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours |
70 |
11/2005 - 5 hours, Pentium-4 1.7Ghz = 8.5 Ghz-Hours |
71 |
@@ -120,26 +121,24 @@ may be broken over the course of 2 years |
72 |
>2000x), then existing checksums do not stand a significant chance of |
73 |
survival in the future. We should thus accept that whatever checksums we |
74 |
are using today, will be broken in the near future, and plan as best as |
75 |
-possible. (A brief review [H04] of the present SHA1 attacks indicates an |
76 |
+possible. (A brief review [H04] of the SHA1 attacks indicates an |
77 |
improvement of ~600x in the same timespan).</p> |
78 |
<p>And for those that claim implementation of these procedures is not yet |
79 |
feasible, see [K06b] for an application that can produce two |
80 |
-self-extracting .exe files, with identical MD5s, and whatever payload |
81 |
-you want.</p> |
82 |
+self-extracting EXE files, with identical MD5s, and whatever payload you |
83 |
+want.</p> |
84 |
</div> |
85 |
<div class="section" id="the-good-news"> |
86 |
<h2><a class="toc-backref" href="#id6">The good news</a></h2> |
87 |
-<p>Of the checksums presently used by Manifest2, one stands close to being |
88 |
-completely broken: SHA1. The SHA2 series has suffered some attacks, but |
89 |
-still remains reasonably solid [G07],[K08]. No attacks against RIPEMD160 |
90 |
-have been published, however it is constructed in the same manner as |
91 |
-MD5, SHA1 and SHA2, so is also vulnerable to the new methods of |
92 |
-cryptanalysis [H04].</p> |
93 |
+<p>Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160), |
94 |
+one stands close to being completely broken: SHA1; and another is |
95 |
+significantly weakened: RIPEMD160. The SHA2 series has suffered some |
96 |
+attacks, but still remains reasonably solid [G07],[K08].</p> |
97 |
<p>To reduce the potential for future problems and any single checksum |
98 |
break leading to a rapid decrease in security, we should incorporate the |
99 |
strongest hash available from each family of checksums, and be prepared |
100 |
to retire old checksums actively, unless there is a overriding reason to |
101 |
-keep a specific checksum.</p> |
102 |
+keep a specific checksum, such as part of a migration plan.</p> |
103 |
</div> |
104 |
<div class="section" id="what-should-be-done"> |
105 |
<h2><a class="toc-backref" href="#id7">What should be done</a></h2> |
106 |
@@ -150,23 +149,30 @@ from Manifest2 files, once all old Porta |
107 |
sufficient time to upgrade. We should be prepared to add stronger |
108 |
checksums wherever possible, and to remove those that have been |
109 |
defeated.</p> |
110 |
-<p>An unsupported hash is not considered to be a failure unless no |
111 |
-supported hashes are available.</p> |
112 |
-<div class="section" id="checksum-depreciation"> |
113 |
-<h3><a class="toc-backref" href="#id8">Checksum depreciation</a></h3> |
114 |
-<p>For the current Portage, SHA1 should be gradually removed, as presents |
115 |
-no advantages over SHA256. Beyond one specific problem (see the next |
116 |
-paragraph), we should add SHA512 (SHA2, 512 bit size), the Whirlpool |
117 |
-checksum (standardized checksum, with no known weaknesses). In future, |
118 |
-as stream-based checksums are developed (in response to the development |
119 |
-by NIST [AHS]), they should be considered and used.</p> |
120 |
-<p>There is one temporary stumbling block at hand - the existing Portage |
121 |
-infrastructure does not support SHA384/512 or Whirlpool, thus hampering |
122 |
-their immediate acceptance. SHA512 is available in Python 2.5, while |
123 |
-SHA1 is already available in Python 2.4. After Python2.5 is established |
124 |
-in a Gentoo media release, that would be a suitable time to remove SHA1 |
125 |
-from Manifest2 files.</p> |
126 |
-</div> |
127 |
+<p>As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms. |
128 |
+In future, as stream-based checksums are developed (in response to the |
129 |
+development by NIST [AHS]), they should be considered and used.</p> |
130 |
+<p>The SHA512 algorithm is available in Python 2.5, which has been a |
131 |
+dependency of Portage since approximately Python 2.1.6.13.</p> |
132 |
+<p>The WHIRLPOOL checksum is not available within the PyCrypto library or |
133 |
+hashlib that is part of Python 2.5, but there are multiple alternative |
134 |
+Python implementations available, ranging from pure Python to C-based |
135 |
+(python-mhash).</p> |
136 |
+<p>The existence unsupported hash is not considered to be a failure unless |
137 |
+no supported hashes are available for a given Manifest entry.</p> |
138 |
+</div> |
139 |
+<div class="section" id="checksum-depreciation-timing"> |
140 |
+<h2><a class="toc-backref" href="#id8">Checksum depreciation timing</a></h2> |
141 |
+<p>For the current Portage, both SHA1 and RIPEMD160 should be immediately |
142 |
+removed, as they present no advantages over the already present SHA256. |
143 |
+SHA256 cannot be replaced immediately with SHA512, as existing Portage |
144 |
+versions need at least one supported algorithm present (SHA256 support |
145 |
+was added in June 2006), so it must be retained for some while.</p> |
146 |
+<p>Immediately: |
147 |
+- Add WHIRLPOOL and SHA512. |
148 |
+- Remove SHA1 and RIPEMD160.</p> |
149 |
+<p>After the majority of Portage installations include SHA512 support: |
150 |
+- Remove SHA256.</p> |
151 |
</div> |
152 |
</div> |
153 |
<div class="section" id="backwards-compatibility"> |
154 |
@@ -238,7 +244,7 @@ Open Publication License, v1.0.</p> |
155 |
<div class="footer"> |
156 |
<hr class="footer" /> |
157 |
<a class="reference external" href="glep-0059.txt">View document source</a>. |
158 |
-Generated on: 2010-01-13 03:27 UTC. |
159 |
+Generated on: 2010-01-31 07:53 UTC. |
160 |
Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. |
161 |
|
162 |
</div> |
163 |
|
164 |
|
165 |
|
166 |
1.5 xml/htdocs/proj/en/glep/glep-0059.txt |
167 |
|
168 |
file : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?rev=1.5&view=markup |
169 |
plain: http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?rev=1.5&content-type=text/plain |
170 |
diff : http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt?r1=1.4&r2=1.5 |
171 |
|
172 |
Index: glep-0059.txt |
173 |
=================================================================== |
174 |
RCS file: /var/cvsroot/gentoo/xml/htdocs/proj/en/glep/glep-0059.txt,v |
175 |
retrieving revision 1.4 |
176 |
retrieving revision 1.5 |
177 |
diff -p -w -b -B -u -u -r1.4 -r1.5 |
178 |
--- glep-0059.txt 13 Jan 2010 03:26:53 -0000 1.4 |
179 |
+++ glep-0059.txt 31 Jan 2010 07:55:45 -0000 1.5 |
180 |
@@ -1,7 +1,7 @@ |
181 |
GLEP: 59 |
182 |
Title: Manifest2 hash policies and security implications |
183 |
-Version: $Revision: 1.4 $ |
184 |
-Last-Modified: $Date: 2010/01/13 03:26:53 $ |
185 |
+Version: $Revision: 1.5 $ |
186 |
+Last-Modified: $Date: 2010/01/31 07:55:45 $ |
187 |
Author: Robin Hugh Johnson <robbat2@g.o>, |
188 |
Status: Draft |
189 |
Type: Standards Track |
190 |
@@ -10,7 +10,7 @@ Requires: 44 |
191 |
Created: October 2006 |
192 |
Updated: November 2007, June 2008, July 2008, October 2008, January 2010 |
193 |
Updates: 44 |
194 |
-Post-History: December 2009 |
195 |
+Post-History: December 2009, January 2010 |
196 |
|
197 |
Abstract |
198 |
======== |
199 |
@@ -39,16 +39,20 @@ is that multiple checksums would be an i |
200 |
could not provably quantify the amount of security this added. |
201 |
The really bad news, is that this position is completely and utterly |
202 |
wrong. Many of you will be aghast at this. There is extremely little |
203 |
-added security in multiple checksums [J04]. For any set of checksums, |
204 |
-the actual strength lies in that of the strongest checksum. |
205 |
+added security in multiple checksums as noted by Joux [J04]. For any set |
206 |
+of checksums, the actual strength lies in that of the strongest |
207 |
+checksum. |
208 |
+ |
209 |
+Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5, |
210 |
+HAVAL-128 and RIPEMD families of hashes. |
211 |
|
212 |
How fast can MD5 be broken? |
213 |
--------------------------- |
214 |
-For a general collision, not a pre-image attack, since the original |
215 |
-announcement by Wang et al [W04], the time required to break MD5 has |
216 |
-been massively reduced. Originally at 1 hour on a near-supercomputer |
217 |
-(IBM P690) and estimated at 64 hours with a Pentium-3 1.7Ghz. This has |
218 |
-gone down to less than in two years, to 17 seconds [K06a]! |
219 |
+For a general collision, not a pre-image attack, since the announcement |
220 |
+by Wang et al [W04], the time required to break MD5 has been massively |
221 |
+reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and |
222 |
+estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to |
223 |
+less than in two years, to 17 seconds [K06a]. |
224 |
|
225 |
08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours |
226 |
03/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours |
227 |
@@ -61,28 +65,26 @@ may be broken over the course of 2 years |
228 |
>2000x), then existing checksums do not stand a significant chance of |
229 |
survival in the future. We should thus accept that whatever checksums we |
230 |
are using today, will be broken in the near future, and plan as best as |
231 |
-possible. (A brief review [H04] of the present SHA1 attacks indicates an |
232 |
+possible. (A brief review [H04] of the SHA1 attacks indicates an |
233 |
improvement of ~600x in the same timespan). |
234 |
|
235 |
And for those that claim implementation of these procedures is not yet |
236 |
feasible, see [K06b] for an application that can produce two |
237 |
-self-extracting .exe files, with identical MD5s, and whatever payload |
238 |
-you want. |
239 |
+self-extracting EXE files, with identical MD5s, and whatever payload you |
240 |
+want. |
241 |
|
242 |
The good news |
243 |
------------- |
244 |
-Of the checksums presently used by Manifest2, one stands close to being |
245 |
-completely broken: SHA1. The SHA2 series has suffered some attacks, but |
246 |
-still remains reasonably solid [G07],[K08]. No attacks against RIPEMD160 |
247 |
-have been published, however it is constructed in the same manner as |
248 |
-MD5, SHA1 and SHA2, so is also vulnerable to the new methods of |
249 |
-cryptanalysis [H04]. |
250 |
+Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160), |
251 |
+one stands close to being completely broken: SHA1; and another is |
252 |
+significantly weakened: RIPEMD160. The SHA2 series has suffered some |
253 |
+attacks, but still remains reasonably solid [G07],[K08]. |
254 |
|
255 |
To reduce the potential for future problems and any single checksum |
256 |
break leading to a rapid decrease in security, we should incorporate the |
257 |
strongest hash available from each family of checksums, and be prepared |
258 |
to retire old checksums actively, unless there is a overriding reason to |
259 |
-keep a specific checksum. |
260 |
+keep a specific checksum, such as part of a migration plan. |
261 |
|
262 |
What should be done |
263 |
------------------- |
264 |
@@ -94,24 +96,35 @@ sufficient time to upgrade. We should be |
265 |
checksums wherever possible, and to remove those that have been |
266 |
defeated. |
267 |
|
268 |
-An unsupported hash is not considered to be a failure unless no |
269 |
-supported hashes are available. |
270 |
+As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms. |
271 |
+In future, as stream-based checksums are developed (in response to the |
272 |
+development by NIST [AHS]), they should be considered and used. |
273 |
+ |
274 |
+The SHA512 algorithm is available in Python 2.5, which has been a |
275 |
+dependency of Portage since approximately Python 2.1.6.13. |
276 |
+ |
277 |
+The WHIRLPOOL checksum is not available within the PyCrypto library or |
278 |
+hashlib that is part of Python 2.5, but there are multiple alternative |
279 |
+Python implementations available, ranging from pure Python to C-based |
280 |
+(python-mhash). |
281 |
+ |
282 |
+The existence unsupported hash is not considered to be a failure unless |
283 |
+no supported hashes are available for a given Manifest entry. |
284 |
+ |
285 |
+Checksum depreciation timing |
286 |
+---------------------------- |
287 |
+For the current Portage, both SHA1 and RIPEMD160 should be immediately |
288 |
+removed, as they present no advantages over the already present SHA256. |
289 |
+SHA256 cannot be replaced immediately with SHA512, as existing Portage |
290 |
+versions need at least one supported algorithm present (SHA256 support |
291 |
+was added in June 2006), so it must be retained for some while. |
292 |
+ |
293 |
+Immediately: |
294 |
+- Add WHIRLPOOL and SHA512. |
295 |
+- Remove SHA1 and RIPEMD160. |
296 |
|
297 |
-Checksum depreciation |
298 |
-~~~~~~~~~~~~~~~~~~~~~ |
299 |
-For the current Portage, SHA1 should be gradually removed, as presents |
300 |
-no advantages over SHA256. Beyond one specific problem (see the next |
301 |
-paragraph), we should add SHA512 (SHA2, 512 bit size), the Whirlpool |
302 |
-checksum (standardized checksum, with no known weaknesses). In future, |
303 |
-as stream-based checksums are developed (in response to the development |
304 |
-by NIST [AHS]), they should be considered and used. |
305 |
- |
306 |
-There is one temporary stumbling block at hand - the existing Portage |
307 |
-infrastructure does not support SHA384/512 or Whirlpool, thus hampering |
308 |
-their immediate acceptance. SHA512 is available in Python 2.5, while |
309 |
-SHA1 is already available in Python 2.4. After Python2.5 is established |
310 |
-in a Gentoo media release, that would be a suitable time to remove SHA1 |
311 |
-from Manifest2 files. |
312 |
+After the majority of Portage installations include SHA512 support: |
313 |
+- Remove SHA256. |
314 |
|
315 |
Backwards Compatibility |
316 |
======================= |