[gentoo-amd64] Re: Re: Re: libungif and giflib conflict.

[Duncan]

Harm Geerts posted <200511021409.38666.harmgeerts@××××.nl>, excerpted
below,  on Wed, 02 Nov 2005 14:09:38 +0100:

> On Wednesday 02 November 2005 05:30, Nick Currier wrote:
>> Looks like that got it guys. Thanks tons for the help.... It seems I broke
>> portage by running only part ~amd64 packages. revdep-rebuild found it but
>> it took twice to fix..... depclean wants to get rid of tons of stuff though
>> so I'm thinking this is a bad idea or I have bigger problems.... Kudos to
>> AMD64 Gentoo for the best support team in open source.
>
> It's normal for a system that hasn't been maintained the way it should ;)

Seconded.

What depclean does is starting from your world file (and including stuff
in your system profile that's not specifically listed in the world file),
figure out what each package listed there needs to RUN (run-time
dependencies, as opposed to compile-time dependencies), BASED ON CURRENT
USE FLAGS, then unmerge other packages as unnecessary.

This is why it's important to try it with --pretend, first, and check the
list for sanity.  There are a number of reasons something might be listed
for removal that needs to be kept.  

1.  Occasionally, there might be something NOT listed in your world file
(and that isn't a dependency of anything else) that you want.  These are
easy enough to append to your world file, removing them and their
dependencies from the list.

2.  USE flags affect dependencies, in many cases.  If you merged something
with one set of use flags, resulting in it linking against a particular
library, then you changed your USE flags, so it would NOT link against it
now, but you still have the old version, linked against that library,
merged, then depclean will want to remove the library (because the current
USE flags say it isn't needed), but doing so will break the currently
merged package with outdated USE flags linked against it.

This is why it's a good idea to run emerge --newuse (-N) --deep (-D)
--update (-u) first.  Doing so should find those currently merged packages
with outdated USE flags and remerge them with current USE flags, thus NOT
linking against that no longer needed library, allowing it to be unmerged
without killing the stuff that formerly depended on it.

It's also why running revdep-rebuild afterward, not just before, is a good
idea.  Doing so afterward catches anything that slipped thru the cracks
the first time.

3.  Note the emphasized RUNTIME above.  The easiest way to explain this
one is with a real example, taken from my experience.  

Example:  k3b is probably the Linux world's best and most recognized all
around CD/DVD burner, ripper, and media transcoder, all in one.  With
various USE flags set, it'll bring in as dependencies transcode and a
bunch of different media codec libraries, that allow transcode to do its
thing, converting from one media format to another on-the-fly.  k3b of
course then uses transcode as one of its tools, to help it convert between
various formats, as well.

The thing is, as the k3b ebuild is currently setup, transcode and a bunch
of these media codec libraries are configured as MERGE TIME dependencies,
NOT RUNTIME dependencies.  Thus, depclean will want to remove them, which
probably isn't what you want, if you had k3b USE flags setup to pull them
in in the first place.  Appending all the affected packages to the world
file will prevent depclean from removing them as unneeded at runtime.

That's probably one of the more extreme examples of where depclean will be
almost GUARANTEED to do the wrong thing, if you don't check what it's
going to do, and do that append before letting it do its thing.  However,
the same principle likely applies in other cases as well.  Even where
packages may not be actually used except to compile packages you /do/ use,
you may want them in your world file, just to prevent depclean from
unmerging them, and therefore having to remerge them the next time the
package you want, that has them as merge-time only dependencies, gets
upgraded.

ALTERNATIVE/HINT:  This is yet another place where FEATURES=buildpkg comes
in handy.  If you have portage automatically creating binary packages of
everything you merge, you don't have to be quite as careful with depclean.
If it removes something you decide you want, simply remerge the binpkg,
avoiding the hassle of having to recompile the package from scratch.  If
you are NOT running FEATURES=buildpkg, yet you aren't sure if something's
needed or not, try using quickpkg to package up the existing package as
currently deployed on your system, before unmerging it.  As above, if you
find you needed the package after all, simply emerge -k it, and the
binpkg you quickpackaged should be remerged, avoiding that extra recompile
time.

> 1. Look over the list of packages that would be unmerged with:
> # emerge --pretend depclean
> 2. Inject the packages that should not be removed in your world file with:
> # emerge --noreplace <package name>

Hmm...  That --noreplace is a new one on me.  I normally simply

echo <category/pkg> >> /var/lib/portage/world

That appends the echoed cat/package to the end of the named file,
accomplishing what we are after.  The only down side to doing it manually
(using echo) is if you typo.  Still, that's not hard to catch and fix, if
depclean still wants to remove the package you (thought you) just added. 
Newer versions of portage (don't believe it's in stable portage yet) even
have an emaint command that removes the bad lines for you.

> Repeats steps 1 and 2 untill you are satisfied.

> 3. clean the remaining packages with:
> # emerge --ask depclean

Note that there's nothing magical about the depclean command itself, other
than the fact that it automatically computes the dependencies and finds
what can be unmerged, for you.  Once it comes to the unmerge itself,
that's really all it does.

Consequently, here, I'll usually do an emerge --pretend depclean, then
simply use the list to add packages to the world file or manually emerge
--unmerge them.  I very rarely run emerge depclean without the --pretend;
that is, I don't often let it do the unmerging on its own.  Rather, I use
it to get a list, and do the unmerging directly, myself.  No big deal,
it's just a matter of style, but I feel more in control doing it myself.
=8^)

One thing about doing the unmerges manually, however, is that if the list
is a bit long, one can unmerge stuff as they decide they don't need it, or
add it to the world file as they decide they do, thus dealing with the
list a bit at a time.  Whittling the list down this way, one can
repeatedly run depclean pretend, addressing just a few packages each time,
until the list gets to a rather more manageable size -- something that
might help if depclean hasn't been run in awhile.

> 4. rebuild packages that were built against older/removed libaries:
> # revdep-rebuild --pretend (part of app-portage/gentoolkit)
>
> In order to keep your system clean in the future upgrade world like
> this:
> 1. upgrade your world, remerge packages that have a changed USE
> flag and upgrade deep dependencies.
> # emerge --ask --deep --newuse --upgrade world
> 2. rebuild packages that were built against older/removed libaries
> # revdep-rebuild --pretend
>
> If you ever remove a package do it like this:
> 1. remove the package
> # emerge --ask --unmerge package
> 2. remove the dependencies but always check the output. other packages
> might use them even though they are not a direct dependency
> # emerge --ask depclean
> 3. rebuild packages that were build against libaries you just removed
> # revdep-rebuild --pretend
>
> This is how I maintain all my systems. If anyone has remarks/tips I'd be
> glad to hear them.

Very well written!  My tips, mainly the FEATURES=buildpkg or quickpkg
thing, to get binary packages and avoid a possible recompile during a
possible remerge, if you are unsure, and the individual package removal
thing, are above, but they are mainly a matter of personal style.  Your
system hygiene routine is top notch!

Two more things...

1.  I already mentioned one of the new features of the new portage
(2.0.53, now in the rc stage in ~arch), emaint, that helps clean invalid
entries out of the world file.  The new portage has another useful new
tool as well, eclean.  eclean is designed to help manage the accumulated
source tarballs in $DISTDIR, cleaning out the ones that no longer match
anything in portage, by default, or if desired, cleaning out all the
tarballs that don't match packages currently merged (older tarballs
for packages with ebuilds still in portage, mainly).  For those using
FEATURES=buildpkg, eclean can manage the $PACKAGEDIR the same way,
deleting binary packages no longer matching anything in portage by
default, or getting stricter and deleting packages not matching stuff
currently merged, if desired.

eclean started out as one of several tools authored and contributed
by Gentoo users, that did much the same thing.  It was selected for
inclusion in portage because it matched the way portage worked already,
and because it did its job well.  For those still running portage 2.0.52
(stable), eclean is available as a separate package, altho it's not in the
tree.  I believe it can be found in the forums.

2.  All this should reemphasize the role of the world file.  It should
list all your "leaf" packages, for the most part, executables.  It should
NOT list "trunk and branch" packages, dependencies of the leaves, for the 
most part, libraries and the like.

The problem with branch dependencies listed in the world file is that when
the leaf dependencies that use them are no longer needed and are unmerged,
depclean still won't try to remove them, because they are listed in the
world file, even tho they are no longer needed.  OTOH, not listing leaf
dependencies in the world file means depclean will try to remove them,
when they are something you WANT to keep. 

Depclean, therefore, is the only tool around that helps to remove stale
dependencies after the packages using them have been unmerged.  Without
it, the leaf packages may be removed from the world file and unmerged, but
the dependencies they brought in continue to hang around, becoming a
breeding ground for possible bugs and security vulnerabilities, in
addition to the space they take up and the the directories they fill.

HINT:  For this reason, here, I've created a set of aliases I use instead
of calling emerge directly.  By default, now, I call emerge --oneshot,
which will NOT add stuff I emerge to the world file.  That way, I can
emerge dependencies to my heart's content, without worrying about having
them clogging up my world file.  In the event I'm actually emerging a new
"leaf" package that's not in the world file yet, I either don't use those
aliases, so it's added to the world file, or (more likely since it's
habit) use the aliases, but then add them to my world file manually, the
next time I run depclean and find them listed.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@g.o mailing list