Re: [gentoo-dev] Re: You currently cannot smoothly upgrade a 4 months old Gentoo system

[Kai Krakow]

Am Mi., 3. Nov. 2021 um 23:15 Uhr schrieb Joshua Kinard <kumba@g.o>:
>
> On 11/3/2021 11:03, Thomas Deutschmann wrote:
> > Hi,
> >
> > it is currently not possible to smoothly run a world upgrade on a 4
> > months old system which doesn't even have a complicated package list:
>
> [snip]
>
> > This is not about finding solution to upgrade the system (in this case
> > it was enough to force PYTHON_TARGETS=python3_8 for portage). This is
> > about raising awareness that Gentoo is a rolling distribution and that
> > we guarantee users to be able to upgrade their system when they do world
> > upgrades just once a year (remember: in my case the last world upgrade
> > is just 4 months old!). If they cannot upgrade their system without
> > manual intervention, we failed to do our job.
> >
> > Situations like this will disqualify Gentoo for any professional
> > environment like this will break automatic upgrades and you cannot roll
> > individual fixes for each possible situation via CFM tools like Salt,
> > Ansible, Puppet or Chef.
> >
> > It would be very appreciated if everyone will pay more attention to this
> > in future. We can do better. In most cases we can avoid problems like
> > this by keeping older ebuilds around much longer for certain key
> > packages to help with upgrades.
> >
> > Thank you.
>
> Actually, it is possible to manage dependency errors like those.  It just
> takes a *lot* of elbow grease, and and long, long time time.  Especially if
> you have museum-grade hardware that these errors are happening on.
>
> For Perl, I've usually just uninstall everything under virtual/* first, then
> try to let it upgrade.  Sometimes that "unsticks" something in perl-core
> enough to let the upgrades apply, pulling back in any needed items from
> virtual/.  If that doesn't solve the problem enough to let emerge do an
> upgrade cycle, I'll try using just the @system target, or start yanking
> things out from perl-core/* one-by-one until emerge shuts up and does what
> it is told.
>
> Also, *always* check for libperl-www being in the package list.  It's
> usually sucked in by way of dev-util/intltool and is responsible for ~35-40
> perl packages alone being pulled in.  If that's in the list, try
> uninstalling just that one, then run a depclean to remove all of its
> dependencies and then see if the upgrade will work.  If the upgrade tries to
> drag intltool or libperl-www back in, use --exclude to hold it out for later.

For me, Qt packages are often a blocker... It seems that on
slot-change, portage isn't able to consider all reverse dependencies
for rebuilt - or rather: It doesn't consider that the old slots
will/can be uninstalled later. I think something similar happens for
perl. Usually, I can solve this by adding `--reinstall-atoms="$(qlist
-IC dev-qt/ dev-perl/)" to the cmdline, then add the remaining reverse
dependencies that need to be rebuilt, too, but portage doesn't
consider for some reason. Usually, that catches more packages than
actually really need to be rebuilt but it cuts down the messy
dependency graph in the error message a lot and enables me to finally
handle it in a sane way. Python upgrades, tho, are a lot weirder and
harder to resolve because it involves portage itself. The latest EAPI
bump was a hard one when I didn't update portage for some time (yes,
it's recommended to do that but that's not always possible for
production containers, and there's also not always time to do that,
and least we are working with containers now with cut down
dependencies, staging and cloning updates with
single-purpose/single-service containers is a lot less headache).

I'm not sure what the problem is here: Somehow portage isn't able to
reach the final result. But then, maybe it should consider not
upgrading all packages at once and even consider less recent package
versions for dependency resolving. This probably would explode the
whole resolver algorithm - and that needs to be optimized properly.
But I'd rather prefer to run portage twice or even more often, if it
at least resolves to an intermediate solution with not always the
latest package versions.

But I'm pretty sure one central problem is portage not always
considering packages for rebuilds properly - and that seems to mostly
happen on slot changes when there are mixed reverse dependencies: some
that depend on a slot, and some that don't. Maybe some resolve
candidates are eliminated just too early from the dependency graph...

That said, I'm usually able to avoid uninstalling packages by using
`--reinstall-atoms`, and sometimes it just needs an `emerge
--deselect` because something stuck it into the world file for reasons
I cannot understand (I'm really picky about what goes into my world
file and try to keep it at the minimum needed).


> That all said, am I alone in thinking that the way Portage emits error
> messages about dependency resolution problems is extremely messy and
> border-line unreadable at times?  The current way it outputs depgraph errors
> feels like something I'd expect from a --debug switch.  We've got a
> reputation for being playful and colorful on the command line with our
> tooling, so I would wonder if that depgraph output couldn't be made to
> look....nicer?

Yep, it's hard to read. It takes a steep learning curve to properly
read those messages and understand what they are telling you, and then
an even steeper learning curve to figure out what action actually has
to be taken. And it doesn't help when portage omits packages with "and
27 other packages with a similar problem" when exactly those are the
ones I'd need to manually stick to the reinstall list. And something
in my head says: "Why doesn't portage just consider those for
automatic reinstall?" - well, probably it did, I'm pretty sure it did.
But something eliminated those too early from the resolve. Increasing
the backtracking usually does exactly nothing except the resolve runs
**a lot** longer.

Regards,
Kai