Re: [gentoo-user] What is the best way forward? - Update 2 - SUCCESS! - CURRENT!!!

[Grant Taylor]

On 2/25/21 5:31 PM, Grant Taylor wrote:
> 10 have git switch to the next day
> 20 emerge -aDUN @world
> 30 assess / deal with masked packages
> 40 goto 10
>
> It /looks/ like things are working.

*TL;DR*

DenverCoder9:  DEAR PEOPLE FROM THE FUTURE ...

This method /does/ work.  I have successfully brought the problem system 
from ~1 year old to ~current (Gentoo portage repo < 24 hours old).

*Speed Bumps*

These were the four things that caused the biggest slow down in this 
process.

1)  Source packages / ebuilds no longer available.
      - I found and downloaded files to DISTDIR.
      - I copied some ebuilds from older versions of portage to my local 
repo.
2)  make.profile not using PORTDIR definition in make.conf.
      - I ran into this while working on October ~ November '20 updates.
3)  PYTHON_TARGETS & PYTHON_SINGLE_TARGET
      - I ran into this after fixing #2.
      - I had to add the following to pull Python 3.6 back in so that 
things would work to add Python 3.7, before allowing the system to 
remove Python 3.6 (again).
           PYTHON_TARGETS="python2_7 python3_6 python3_7"
           PYTHON_SINGLE_TARGET="python3_7"
4)  Firefox & Thunderbird 68 disliking rust ≈ 1.48.
      - I had to give up on retaining version 68 of Firefox and Thunderbird.
         - The loss of some important extensions still really hurts.

*How*

The high level process that I used is a very close superset of what I 
hypothesized.

10 have git switch to the next day
20 emerge -DUN @world
21 emerge --depclean --verbose n
22 emerge @preserved-rebuild
23 revdep-rebuild
30 assess / deal with output from steps 20-23
40 goto 10

Steps 21-23 added mid-stream to make comparison to previous message simpler.

All of these steps were in a function, `e3` (see attached file), which 
relied on one variable, `d`, the count of how many days to go backwards 
and set the date (`D`) to that everything should act on.

Aside:  The next version of e3 would probably store `d` in a file and 
subsequently re-load it from said file on each invocation.  Thus 
eliminating the reliance on the environment variable.  I would probably 
store this file in /var/tmp as /tmp and /dev/shm are cleared on boot.

After gaining enough trust in the overall process, I ended up running 
the following while loop:

    while e3; true; done

This allowed the system to stay busy emerging things up to the point 
that something failed and needed attention.

*Setup*

I did a `git clone` of the Gentoo portage repo.  Currently ~6 GB.

I then created the branches in the git repo with the following command 
(from inside of the git repo directory):

    for ((age=1; age<1024; age++)); do eval $(printf 'git log 
--pretty=format:"%%H %%cd" --date=format:%%Y-%%m-%%d\ %%H:%%M:%%S 
--after=%s --before=%s | fgrep -m1 %s' $(date +%Y-%m-%d -d "$(($age + 
1)) days ago") $(date +%Y-%m-%d -d "$(($age - 1)) days ago") $(date 
+%Y-%m-%d -d "$age days ago")) | read hash date time; time git checkout 
-b $date $hash; done

Basically, this command starts at current; `stable`, and finds the first 
(most recent) commit for a given date and creates a branch, and works 
backwards for however many days configured; 1024 in the example.

*Miscellany*

I did `emerge -e @world` 3~5 times throughout the process just to make 
sure that everything was consistent.  I will do this once more tomorrow 
after a full backup runs tonight.

I did end up removing a small list of packages that were blocking emerge 
in one way or another.  --  I decided that removing them to allow emerge 
to complete on it's own accord was more expedient than fighting them at 
the time.  I will re-add them as necessary.

  - net-firewall/nftables
  - net-fs/ncpfs
  - media-gfx/gimp
  - dev-python/pycairo
  - dev-python/fido2
  - net-analyzer/scapy
  - app-crypt/yubikey-manager

Some of the packages were subsequently pulled back in.

I did run into a bug with app-misc/pax-utils where I needed to add 
"-seccomp" for the package to be able to move forward.

I also did the /usr/portage to /var/db/repos/gentoo et al. migration.

"repo" can be ambiguous when there talking about both "Gentoo portage 
repo" and "git repo".  Especially when the latter is managing the former.

The following packages take what seems like  F O R E V E R  to emerge:

  - gcc
  - rust
  - Firefox
  - Thunderbird

Link - xkcd - Wisdom of the Ancients (a.k.a. DenverCoder9)
  - https://xkcd.com/979/

*Summary*

Yes, there are probably faster and / or more efficient processes to get 
a Gentoo system that's ~1 year behind caught up to current.  But I did 
learn some things along the way.  --  I tried to outline the toe 
stubbers so others can avoid them.

Ultimately, I believe I have done in the last 11 days what would have 
been done over the course of the last ~year.  Even 11 days is longer 
than necessary as I started with the while loop after getting to January 
of this year.  In hindsight, I believe I could have used the while loop 
all along.

I hope that I have included enough details for others to be able to 
reproduce this process if they need to.

Now it's time to repeat this process to bring the two companion systems 
up to date.



-- 
Grant. . . .
unix || die