Re: [gentoo-dev] RFC about another *DEPEND variable

[Mike Frysinger]

On Wednesday 27 September 2006 03:54, Brian Harring wrote:
> On Wed, Sep 27, 2006 at 02:24:41AM -0400, Mike Frysinger wrote:
> > as i said, if you have changed ABI without an ABI bump, then the upstream
> > package maintainer is screwing everyone who uses the package, not just
> > Gentoo ... so perhaps we should be talking to them to get the situation
> > resolved for all consumers, not just Gentoo
>
> Happens however; afaik, we also weren't monkey patching ssl in the
> past to preserve the abi either so it still is valid (although rare if
> upstream is behaving) scenario.

we've never monkey patched ssl so i dont really know what you're referring to

> > > Bleh, this is getting back to exactly my point that it's unbounded
> > > resolution.  To support this, every step of execution would require
> > > scanning for dangling nodes to punt; aside from invalidating -p's
> > > output it *still* is a collision-protect hit.
> >
> > when the package upgrade detects an ABI change you recalculate the
> > packages that need to be rebuilt ...
>
> Reiterating, this screws over any form of up front calculation;
> dialup users/per minute connections can't rely on emerge -f to
> actually fetch all involved sources, -p results ain't guranteed
> valid, parallelization must now lock at each threads merge on the off
> chance a recalc is forced.

it does indeed prevent full up front calculation.  we can always make the 
behavior configurable; revdep on the fly or allow people to break it up.  the 
key being that their system will still continue to function as the ABI lib 
has been preserved automagically

> > dangling nodes get recorded in the new package and considering these old
> > files are not detrimental to the health of the system, you can do such a
> > cleanup once at the end of the emerge
>
> It's not orphaning files, but your scheme doesn't work for any form of
> interruption; failed builds, user intervention, etc, they all can
> leave the lib stuck in the new contents.

huh ?  i outlined in a previous e-mail how by simply ordering the operations 
sanely, there is no race condition

> Dealing with that possibility means the manager has to maintain on
> disk a list of the refcount of each dangling libs to decrement,
> unmerge has to modify said list, etc.

which is already being done in the NEEDED file ... funny how unpainless it is 
to generate that file

> > > It also involves changing vdb nodes from "installed and usable" to
> > > "installed/usable" or "lingering"
> >
> > no ... the old versions get merged into the new one as their existence is
> > purely hidden
>
> How exactly are they going to be hidden?  A new file?  If so,
> backwards compatibility for vdb for transitioning in such a support
> has to be addressed.

purely hidden from the standpoint of any new package trying to use it ... 
since you're only installing the runtime ABI library, you cannot link against 
it or utilize it any way other than existing files

> > > Tracking BINCOMPAT should *not* be that hard.
> >
> > it's one more thing to keep track of and considering all of the
> > possibilities i outlined, a single maintainer can easily lose his sanity
> > ... or you force wasted rebuilds on people when it's only needed in some
> > circumstances
>
> How exactly is this forcing wasted rebuilds?  You're stating that
> maintainers are going to bump it willy nilly.  As I said, it is a key
> that would be bumped *as needed*, and would only affected pkgs that
> specified that node as a binding dep (specially marked atom).

no, i mean for example scenarios where a package provides more than one 
library (say libfoo and libbar) and only one of those changes ABI values (say 
libfoo.0 -> libfoo.1).  if another package links against just libbar, you've 
got a pointless rebuild.

> Seriously, maintainers ought to know *now* when they're forcing a
> revdep on folks systems, I'm not seeing the massive overhead from
> pushing that info into a var, nor am I seeing mass forced useless
> rebuilds.

there's a ton of things maintainers ought to know ... pretty soon our package 
maintainers are going to have to be gods if they want to write an ebuild as 
they're going to have to know every detail about the package inside and out

> Realistically, just the same as the NEEDED solution has holes, the
> maintaining dev can screw up and miss a BINCOMPAT bump.  Difference is
> that they can do something for BINCOMPAT; hell, QA checks can be
> automated to catch missing BINCOMPAT bumps.

what's the difference ?  in my scenario they dont have to do anything because 
the bump would have been caught automatically ?

> KEYWORDed arches bit, bit unlikely that the underlying arch is
> actually going to screw with the soname, thus I'd want actual examples
> backing it up.

how about libc.so from glibc and libgcc.so from gcc ?  or would you like other 
packages ?

> Besides, again, for keywording, the dev *should* be compiling it, so
> there is a way to catch it :P.  A revbump isn't going to break things
> unless the dev is introducing something intrusive, minor version bumps
> (1.1.0 to 1.1.1) shouldn't be again, introducing anything intrusive;
> regardless, dev should know the high level affects of their change

uhh, there is no such requirement at this time for revbumping on different 
arches ... currently we say the maintainer tests for his arches and leaves 
all the others as ~arch

> Inducing a revdep-rebuild is definitely one of those high level
> changes they should be aware of *prior* to the bump.

and yet we look at our history of so many packages being bumped with ABI 
changes and users having broken-as-shit systems ... i'm accounting for the 
worst; not hoping for the best

> > or dig
> > through the source code line by line and hope to catch all such cases by
> > hand/eye ?
>
> Bit of FUD here, although I spose I'll just point out that if so's
> change as massively as you're implying above, the affects on -p are
> that much worse.

awesome, mark something you disagree with as FUD, problem solved.  my point is 
that you can never know completely for sure the behavior of a package without 
digging through the entire source code.

> bind the checks in as a QA measure, enabled via FEATURES=stricter,
> used to maintain a metadata var.

a lot of things fail already with FEATURES=stricter ... too bad we dont have 
per-package env var support as then maintainers could just flag all their own 
packages as stricter without wanting to shoot themselves due to everyone 
else's package failures

> Literally, pkg x version y forced a rebuild.  revdep is annoying, but
> stats would be useful to actually evaluate the seperate proposals;
> related, getting stats for how often the various updaters were
> required to be ran for a particular pkgs upgrade would be useful in
> evaluating that particular gap NEEDED has.

any openssl version bump where the numerical value changes ... only the letter 
updates preserve ABI compat (0.9.7[a-z] are compat)

expat-1.x -> expat-2.x

iirc, tcl-8.3.x -> tcl-8.4.x

readline-4.x -> readline-5.x

ncurses-4.x -> ncurses-5.x
-mike