Re: [gentoo-dev] Manifest2 hashes: validation of single hash per MANIFESTx_REQUIRED_HASH

[R0b0t1]

Hello friends!

On Wed, Nov 15, 2017 at 3:02 PM, Robin H. Johnson <robbat2@g.o> wrote:
> Replying to your original question here, to repeat the answer I emphasised
> before, along with significantly more detail in the history of Portage hashes
> (pulled from my notes back to GLEP57 and some minor updates).
>
> On Wed, Nov 08, 2017 at 12:57:49PM -0600, R0b0t1 wrote:
>> These posts are concerning because it looks like someone became stir
>> crazy and invented a problem to solve. The changes proposed to date
>> have remained poorly justified, and no one has addressed the concern
>> that multiple hashes *is* actually more secure.
>>
>> If it was deemed necessary at one point, what justification was used?
>> I.e. https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence.
> On Wed, Nov 15, 2017 at 11:47:41AM -0600, R0b0t1 wrote:
>> Does the existence of a decision mean I would need to contact the trustees
>> if I feel the changes have not been adequately justified?
>
> In GLEP59, I referenced a paper by Joux [J04], in which it was shown that a
> concatenation of multiple hashes is NOT much more secure against collisions
> than the strongest of the individual hashes.
>
> That was cited as original logic in GLEP59 for the removal of SHA256 (that
> removal was never implemented). WHIRLPOOL & SHA512 were kept out of an
> abundance of caution at the time, mostly to implementation bugs in hashes (as I
> have referenced in the related threads since).
>
> Your logic regarding removing something you think I don't understand is wrong
> (Chesterton's Fence):
>
> If you dig in the history of Portage, you will see that it's always been valid,
> to have just a SINGLE hash for each file in a Manifest.  Required hashes has
> NEVER contained more than one hash.
>
> If multiple hashes are present, then Portage will validate all of them, but a
> potential attacker can still modify the Manifest and have only a single hash
> listed.  Exactly which hash MUST be present has changed over time.
>
> Manifest1 is very old, and was stored in $CAT/$PN/files/digest-$P
> Manifest2 is the current $CAT/$PN/Manifest (and soon in more locations per MetaManifest).
>
> 1999/xx/xx: Portage starts with Manifest1 format, MD5-only (CVS)
> 2004/08/25: Portage gets SHA1 support in Manifest1, but is problematic, SHA1 generation manual only.
> https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-src/portage/pym/portage_checksum.py?revision=1.1&view=markup
> https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-src/portage/pym/portage.py?r1=1.485&r2=1.486
> 2005/12/19: Portage Manifest1 supports MD5,SHA1,SHA256,RMD160, but still requires only a single hash present. Generates MD5+SHA256+RMD160.
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=cd3e3775966a9f58aebb91f58cbdb5903faad3de
> 2006/03/24: Manifest2 introduced.
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=f993747ca501e8a70d6f6174711149a172cfc3c2
> 2007/01/20: MANIFEST2_REQUIRED_HASH introduced, SHA1, it must be present & pass
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e768571187d1655fbb558c23d61fa2983e48e411
> 2007/12/18: MANIFEST1_REQUIRED_HASH introduced, MD5, it must be present & pass
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=d9b10deaa03ce174d5ccc3b59c477549ad87e884
> 2008/02/28: Manifest1 support dropped.
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=66940e1f2f0549ee8f01dad59016e168105e193d
> 2011/10/02: GLEP59 implemented, MANIFEST2_REQUIRED_HASH changes to SHA256
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=c8cd3a985cc529299411d7343a11004b7d1330ef
> 2017/06/15: MANIFEST2_REQUIRED_HASH changes to SHA512
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e6abcc0b7cbdca481862a5c7cca946c01c471ffb
>
> [J04] Joux, Antoie. (2004). "Multicollisions in Iterated Hash Functions - Application to Cascaded Constructions;"
> Proceedings of CRYPTO 2004, Franklin, M. (Ed); Lecture Notes in Computer Science 3152, pp. 306-316.
> Available online from: http://web.cecs.pdx.edu/~teshrim/spring06/papers/general-attacks/multi-joux.pdf
>

This is the information I was looking for, thank you. I feel that the
matter has been adequately explained. I apologize for missing your
response.

The paper gives a counter intuitive result, so I suspect I will have
to spend more time with it.

Cheers,
     R0b0t1