Re: [gentoo-portage-dev] Re: [gentoo-dev] digest + manifest = new file format

[Jason Stubbs]

On Friday 05 August 2005 11:42, Alec Warner wrote:
> Whoopse, should have gone to gentoo-portage-dev, my bad ;)
>
> Alec Warner wrote:
> > This is basically a resubmission of Genone's file format merger [1].
> >
> > The code to handle digests and manifests is currently being designed
> > and rewritten.  As such we need two things from the developer
> > community. One is a decision of what the new format will entail.  Two
> > is a firm agreement on the migration plan.
> >
> > 1.  The file format is specified in Genone's original mail, but I'll go
> > over it again to save you the extra linkage ;)
> >
> > Inside of the digest-manifest we have:
> > EBUILD Filename SIZE 1234567 MD5 A4FD085FF SHA-256 A7439FDB1
> > <type> <filename> <size-tag> <size> <hash-tag> <value> <hash-tag>
> > <value>....
> > Where type is the type of file (DIGEST, SRC_URI, EBUILD, PATCH, etc)
> >
> > *Note, that the size-tag was added to make the human parsing easier.

That's incorrect. It was added because there is nothing special about size 
other than it being another way to improve your percentage of chance that 
the file you have matches.

> > 2.  If you didn't notice by now, this format breaks versions of portage
> > that don't support it ( ie 2.0.X ).  This leads us to a few options as
> > users migrate to a version of portage that supports the new file
> > format.
> >
> > 	A.  Write some code into the final stable version to do both kind of
> > formats and/or use the new format style but fall back to the old style
> > ( since at release time the new style format probably won't be in the
> > tree yet ).  Either way, the last release of 2.0.X should work with the
> > new file format in enough of a fashion to not break.  This means that
> > when the old manifest + digests are combined in the tree, most users
> > should be ok even if they aren't upgraded to 2.1.X.
> >
> > 	B.  Have a migration time where both formats are in the tree.  For a
> > while the tree will be larger, and many people will have issues with
> > this.  However, old portage will use the old files, and new releases
> > will use the new files.  Then after a period of however long ( 6
> > months? ) announce loudly beforehand that portage-2.0.X will no longer
> > be supported and pull the old digests/manifests out of the tree.
> >
> > 	C.  The Carpaski way ;)[2]  Add support for the new format in 2.1,
> > while also adding support for current format.  Force everyone to
> > upgrade to 2.1.X, while announcing that 2.0.X will not work in the
> > future.  When we have confidence that most users are upgraded, pull the
> > old format and add the new format to the tree.
> >
> > 	Problems with all of these include the same problems as the cascaded
> > profiles, some goofball doesn't upgrade for a year, syncs with new
> > digests...how does he get his portage upgraded?  An upgrade path should
> > be provided and documented in this case.
> >
> > The best route is probably some combination of the above.
> > Pre-emptively add support for the new format to stable, while
> > announcing the death of the old format after 2.1's release.  When most
> > users upgrade normally to the latest 2.0.X series ( perhaps fearing the
> > changes of 2.1.X ) they will gain support for the new file format (or
> > if not support, merely a working portage instead of broken).  We should
> > catch the majority of users who upgrade with either the last 2.0.X
> > release or the new release of 2.1.X.
> >
> > Suggestions, criticisms, etc... welcome.

This stuff is all going into the Manifest rather than per ebuild digests, 
right? Portage already ignores any line in Manifest that does not begin 
with MD5. Manifests can be easily bypassed with FEATURES="-strict" anyway. 
The only real issue is SRC_URI files. An upgrade path can be made available 
by maintaining the digest.* files only for portage.

There are/will be several more issues like these. The idea is already to 
have an ebuild format tag that portage will either be able to handle or 
not. How about just patching stable to not allow merging of any ebuild that 
has the tag defined and requesting the user to upgrade?

In this case, the new portage would need to support both formats but the 
tree wouldn't require them both formats. The new commit tool would generate 
an old style manifest/digest for old ebuilds and new style for new ones.

--
Jason Stubbs