Re: [gentoo-portage-dev] tree dependency check

[Brian Harring]

On 3/29/06, Marius Mauch <genone@g.o> wrote:
> Marius Mauch schrieb:
> > So after manifest2 is in, I'll revive the other issue that IMO is a
> > requirement for 2.1: enforcing dependencies needed to use the tree (see
> > old threads or glep44 for reasoning). A patch for that is available at
> > dev.gentoo.org/~genone/patches/treedeps.diff. Unless somebody objects
> > I'll add that somewhere next week.
>
> Ok, from a discussion with Zac and a few others it seems that we should
> add a layer of indirection, so instead of specifying atoms the tree has
> a freeform version identifier, and portage can use it to look up the
> required atoms by using a (remote) mapping file.
> This new approach is implemented in
> dev.gentoo.org/~genone/pacthes/format-check.diff (it's basically a
> combination of strategies D and E in d.g.o/~genone/docs/treedeps.txt)
>
> This new patch also has a few other improvements:
> - checks overlays too
> - at sync time first check the tree format of the remote tree, if we
> can't handle it the sync is aborted
> - if no format specification is found warn the user about it (with the
> intention to fail in a future version)
>
> New intended merge date is this weekend or beginning of next week, so if
> you have any objections, make them now.

In doing compatibility code, should wait a bit to make sure it's sane
and that there are no issues _prior_ to deployment- rushing it (3 days
notice) kind of conflicts with that requirement ;-)

Mind you this is from looking at the code (not running it), so verify
it, but here goes...

Implementation issues...

1) format_check should just return the results (supported, not
supported, need xyz) instead of printing to the term.  Remember this
code may be triggered by gui apps.  This also means it shouldn't be
abused to check what's been handed in via commandline to know what to
exempt, should just return what is required.

2) code isn't root aware.

3) no snapshot integration.

4) code invalidly assumes that all later version of the pulled atom
from vdb will work.  Format version 1 may not be supported by portage
2 years down the line, this code assumes all portage versions from
that point on support that format- that is a recipe for bloat/borkage;
this is a bit odd of a check/attempt considering the url crap to try
and get the appropriate atom.

5) Introduced (in the urlopen call) another KeyboardInterrupt
swallowing except clause.  Don't Do That. :)

6) This breaks _all_ syncing for users who have overlays but lack the
format versions file.  That's a massive no go, you don't break
compatibility introducing compatibility checks (nor do you piss off
several thousand overlay using users for questionable gain).

7) even cooler, say you're running max visible portage, and using an
overlay that lacks a format_version file.  With the vdb portage
lookup, it'll tell the user that they need a version later then the
max version.  Nice way to get people to test package masked portage's,
but it still is wrong.

8) (minor) output of todo is going to be fugly if anyone uses actual
atom constructs, boolean ORs fex (the print implicitly assumes it's
just a list of atoms without any boolean constructs).

9) the attempted check to see if a pkg is in the passed in myfiles
won't work if myfiles holds atoms; eg,
myfiles, pkg = [">=sys-apps/portage-2.0.54"], "sys-apps/portage"
assert pkg not in myfiles

Perceived design flaws...

1) pollution of the rootdir of PORTDIR.  To head of the "it's
subjective", aside from this repo id will need to be pushed into the
tree format also, and any files the tree format version requires
*should* be in a subdir.  so... basically this single file is going to
be sitting in rootdir while all other repository metadata is stuck in
a subdir.  Seems rather daft to have it shoved in it's own location-
compatibility code _does_ have certain hardcoded assumptions for
things to work, hardcoding looking in a subdir is no different then
hardcoding no subdir (only difference is the subdir keeps things
contained/clean).

2) It's overengineered.  There is _no_ reason to hit up a webserver
just to get atoms; that data can be bundled in the tree in a seperate
file.  As is, this breaks users who sync without a connection without
any gain.  Realistically, I'd be surprised if any alt package managers
go this route (I know I won't be hitting a webserver up for pkgcore).

3) What the portage project thinks a repo tree needs does not map to
what my tree may need.  Clarifying, format 1 specifies portage xyz and
bash-3 (ebuilds in the tree use bash regex).  My personal tree needs
portage xyz (manifest/layout changes), but requires just bash-2.  With
the central db approach, portage will assume my tree is valid via the
version #, and if the number differs, it'll assume that I require
bash3 when in reality, my tree is bash-2 and up.  This points to why
the format -> depends mapping should be bundled with the tree.

4) portage-2.0 portage-2.1 are bad names for tree formats.  Back to
the EAPI int discussion, but I really don't see any gain in string
names, regardless the 2.0/2.1 sucks as a descriptive name (subjective
I realize) :)

Short version- this isn't ready for prime time and (personally) I
think it needs to be rethought.  At the very least it needs to _not_
cause user visible uglyness on enabling the compatibility checks (need
a solution for transitioning from unversioned repos to versioned
repos).  Requiring folks to jam a file with a random string in it in
each of their overlays also sucks, ways to make that less sucky would
be advisable.

Not opposed to the concept, but think this needs a fair amount of use
case planning- the bash-3 example folks *will* try jamming in the bash
dep fex (spyderous already wants the go ahead for using regex), and
while the intention seems to be versioning the tree format, the bash
requirement for the tree is also there.

So... -1.
~harring

-- 
gentoo-portage-dev@g.o mailing list