Re: [gentoo-user] Pick your hypothesis:

[Rich Freeman]

On Thu, Jan 24, 2019 at 7:55 PM Neil Bothwick <neil@××××××××××.uk> wrote:
>
> On Thu, 24 Jan 2019 19:18:17 -0500, Alan Grimes wrote:
>
> > Since 2004, I have been using "emerge --sync" to update my portage tree.
> > I understand that it accesses a local list of mirrors and then runs
> > rsync against one of those. On spinning media hard disks it is helpful
> > to pre-cache the portage tree as shown in my scripts, on SSD systems, it
> > is simply harmless. I refreshed my mirror list and selected either HTTP
> > or RSYNC mirrors from across the USA.
>
> I doubt that there is an advantage in pre-caching, all you are doing is
> taking time from the sync process and adding it to another process, but I
> also doubt that has anything to do with your problems.
>
> > The problem is NOT with emerge in that the files that it complains about
> > not being present are in fact not present.
>
> That's the rub, they are present here, so something is interfering with
> your downloading of these patch files. Do you have any entries in
> make.conf that would modify rsync behaviour? Try
>
> emerge --info -v | grep -i sync
>
> Also, what do you have in /etc/portage/repos.conf/gentoo.conf?
>

Some kind of rsync issue (assuming rsync is even being used) seems
likely.  It is a bit odd since I'd think that rsync would download the
missing files at about the same time that it downloads the manifests -
I assume that it goes in directory order.

If you have some kind of environment variable or make.conf option set
that overrides rsync defaults maybe that is causing rsync to not
actually sync the tree?  Deleting /usr/portage and re-syncing seems
like an obvious way to wipe the slate clean.  Other local issues could
be anything that might cause rsync to terminate prematurely (RAM,
ulimit, whatever - obviously unlikely).

Knowing what mirror is in use may also be helpful - could be a bad
mirror or something.  What server/IP/etc is being output in the emerge
--sync output?

If you're syncing with git and manually modify stuff in the local repo
that can cause issues since git will start getting merge conflicts.
rsync just overwrites your repo with the Gentoo one and wipes out your
own changes.

If you use emerge-webrsync then the sync operation itself should use
gpg signature checking and that should rule out server issues.  Ditto
if you sync with git with the appropriate settings, though git also
has internal hash consistency checking (that doesn't prevent
intentional tampering but should catch data transmission issues).

The suspicion of an error by a Gentoo developer is a possibility, but
these are pretty rare and usually get fixed very quickly.  If you sync
from one of the stable mirrors in git those are all screened by CI
which eliminates these sorts of missing file issues (granted, in git
we don't track FILESDIR in manifests anyway).  If this is happening it
would be trivial to detect on the various online git repositories - so
checking these for the error before pointing fingers is appropriate.
Typically these sorts of errors are caught by bots and result in
plenty of internal shaming already, which is why it is likely a
problem like this would be short-lived.

-- 
Rich