[gentoo-dev] Re: revbumping ebuilds after USE dependency changes

[Duncan]

Michał Górny posted on Sun, 28 Jul 2013 11:11:13 +0200 as excerpted:

> With a proper design, you have two 'repos': one with ebuilds,
> and the other consisting purely of installed packages (vardb/system).
> What's important, per definition vardb is self-satisfactory. That is,
> dependencies of every installed package are supposed to be satisfied by
> installed packages purely.
>
> Now, what portage does is implicitly applying _some_ of the metadata
> from the ebuild tree to vardb without rebuilding the package. In some
> cases. As an effect, vardb is no longer self-satisfactory,
> and represents something between the package that was built and the
> current ebuild.
>
> Ciaran has already elaborated a bit on the potential issues. It gets
> most dangerous when you create some meaningful changes without a
> revbump. I'll give you a simple example that I can think of.
>
> Say, you fix a semi-build-time issue of linking against unnecessary dep.
> Users who build the ebuild from now on benefit by having less deps. The
> dep is less problematic than rebuilding the package, so users who built
> it before prefer to wait for next version.
>
> But in this case, portage may implicitly update the deps from ebuild
> without rebuilding it. This means that users who still link against the
> dep, may end up with the dep removed and program broken.

This is an extremely good explanation (much better and shorter than I 
would/could have done, I believe).  Thanks.

The one thing it's missing, and that's more on the practical than 
theoretical end I believe the explanation was targeting, is that these 
days, portage's depclean has an additional level of caution/safety built-
in, that in practice limits the damage quite a bit from what the above 
theory would predict, very likely in response to complaints about that 
very problem.

I haven't checked the details and depclean does run far faster than 
revdep-rebuild so whatever it's doing isn't as thorough, but depclean now 
does at least some actual on-system checking before removing a package, 
and will refuse to remove a package it otherwise would (no database deps 
requiring it so the database says it's fine to remove) with an 
explanation of what's actually linked against it, if depclean's pre-clean 
scans indicate that removing the package would otherwise leave an 
unsatisfied/dangling linkage, asking you to rebuild the depending package 
first to remove that in-practice-but-not-in-database dependency.

This behavior can actually be a bit frustrating if the depending package 
in question has an "automagic" dependency that will link against a 
package if it finds it installed, but doesn't declare it as a dep, as the 
requested rebuild won't solve the undeclared dependency in that case, so 
the depended package must be removed manually (using emerge --unmerge, 
which doesn't do this scan and has a nice scary warning to that effect) 
and the automagic-depending package rebuilt AFTER the depended package as 
been removed.

That's actually how I ended up finding out how well depclean's scans 
work, since it said to rebuild a package that didn't declare a 
dependency, which I did, but that didn't solve the problem as it was an 
automagic dependency that was linked in as long as the files were found 
on the system to link against.  The gentoo automagic-dependency bug I 
filed was initially closed as invalid, too, because the gentoo package 
maintaining dev wasn't in fact aware of depclean's then rather new 
ability to catch such problems, either.  But I reopened, mentioning 
depclean's (then) new scanning and asking him to actually try it and to 
consult with Zac if he had questions about the (then) new behavior, which 
he evidently did, as the bug was fixed properly, automagic dependency 
converted into a USE flag covered dependency, with a comment to the bug 
thanking me for re-opening and alerting him to the problem. =:^)

Altho I believe it's limited to elf-dependency checking and arguably 
isn't the best theoretical solution, in practice depclean's solution is 
rather nice, since it catches both dynamic-dep problems like those in 
this thread, and automagic-linking issues that no pure database solution 
could catch, since by definition, automagic dependencies are not declared 
in the database, or they'd be declared dependencies not automagic 
dependencies.  (By the same token, it should catch dependencies for 
independently compiled and installed packages that are obviously not in 
the database except possibly via package.provided, as well. =:^)

But I do believe depclean's scan won't catch python/perl/ruby/shell/etc 
dependencies, only elf-binary.  That remains a problem, but one that for 
many gentooers anyway, isn't actually too big a deal in practice.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman