Re: [gentoo-dev] proposed md5sum change

[Paul de Vrieze]

On Thursday 12 June 2003 09:53, Evan Powers wrote:
> On Thursday 12 June 2003 02:56 am, Seemant Kulleen wrote:
> > Anyway, the current approach keeps it simple in that the md5sum is off
> > the *item(s) that is/are downloaded*.  The first reason I can see is what
> > I stated above.  There are other reasons I can see as well.  You know,
> > immediately upon fetching the set of source items that they are bad.  So,
> > no disk i/o or cpu cycles are spent in the unpacking; and no potentially
> > nasty code is even untarred on the system, yet.
>
> Well, I can think of a way to address this part of the issue, anyway.
>
> * The portage tree still has MD5 digests for the item(s) which is/are
> downloaded.
>
This would make things easy for most people, and would save also lots of cpu 
cycles.

> * After downloading, emerge executes a (user specified?) program on the
> newly downloaded file. This program applies some transform to the file;
> maybe it decompresses whatever format the file is in and re-compresses it
> with bzip2, or maybe it only format-shifts files which are over a certain
> size threshold, whatever.
>
Allways fun, maybe we would also have a list of allowed/blacklisted extensions 
like RPM files which are allready compressed. (Or openoffice files/jar-balls 
which loose validity if compressed in a different format, but could be 
recompressed in zip format)

> * Next, emerge adds a new record to a database (text, one record per line,
> for example) somewhere in /var. This database has the original name of the
> downloaded file, the original MD5 digest, the new name, and the new MD5
> digest.
>
Maybe extending the serverside digest to also include an unpacked digest 
(where applicable) would be smarter in validation of patch based and oddball 
cases. (This doesn't mean that client-side digesting isn't useful, it is for 
not having to unpack first).

> With infrastructure like this you could even add more interesting
> functionality to portage pretty easily. Like maybe the transform program
> uploads the file to the corporate internal FTP mirror, and the database
> maps the original name to the URI which locates it.
>
> Or, if emerge exported sufficient context to the transform program, you
> could fix the case where a particular braindead package is available only
> as package.tar.gz, not package-version.tar.gz. The transform program would
> add the version to the filename, and the database would be allowed to have
> multiple entries for each original file name (provided they had different
> MD5 digests). Then emerge would just pick the record with both the desired
> original name and the desired original MD5 digest.
>

I do not know whether package (name) transformation is the same thing, but I 
support it fully. I believe that name transformation should also be specified 
in the ebuild. That would lead to a variable like:

TRANSFORM="foo.tgz:foo-0.0.1.tar.gz 
bar-0.1Beta1.2.tar.bz2:bar-0.1_beta102.tar.bz2"

Which would automatically transform filenames. This would also apply to the 
mirrors, so portage needs to be changed to try to fetch the transformed name 
from the mirrors while trying to fetch the original name from the source. I 
do believe though that name transformation is a separate issue.

Paul

-- 
Paul de Vrieze
Researcher
Mail: pauldv@××××××.nl
Homepage: http://www.cs.kun.nl/~pauldv