Re: Re: [gentoo-user] local mirrors

[Rich Freeman]

On Fri, Apr 26, 2019 at 1:42 AM <n952162@×××.de> wrote:
>
> What I'm hoping for, though, is a configuration flag that causes any of:
> - inhibits a theoretical traversal of repos.conf databases (e.g. also /usr/share/porrage/...)
> - inhibits trying to upgrade beyond what's already installed in the mirror server

There is no traversal.  The repository is only obtained from a single place.

It is configured in repos.conf.  Whatever that is pointed at is where
your repository will come from.  If you point that at your mirror,
then your host will never go out to the internet to update the
repository, and the list of available packages will be whatever is on
that mirror.  This is the way most people using Gentoo in a datacenter
do things, for all the reasons you might think.

This is distinct from the distfiles, which are the actual source code
tarballs used to install packages.  That is configured in make.conf
using the GENTOO_MIRRORS setting.  That IS traversed - portage will
start at the first URL and work its way down the list until the file
is found, and if all are exhausted then it will go to the original
upstream source of the file if there is one.  I don't think there is
any way to get portage to not traverse this entire list, and the
SRC_URI in the ebuilds themselves.  Though, I'm not sure why you'd
want this process to stop if it can't find something unless you're
really concerned with bandwidth - the files are all protected by
hashes stored in your repository so no matter where the file comes
from it won't be used if it isn't the right file.

So, to sum up, there are two types of mirrors:
Repostory mirror, set in repos.conf sync-uri setting
Distfiles mirror(s), set in make.conf GENTOO_MIRRORS setting.

Note also, all distfiles mirrors are checked for all repositories, if
you have more than one.

> >
> > Thank you for the explanation.  I wonder what the local mirror page means when it talks about saving bandwidth.   What *does* it serve if not the distfiles?

The repository.  IMO the bandwidth savings would be minimal unless you
had a lot of hosts, compared to mirroring distfiles, which is
problematic as you seem to get.

> And when /etc/portage/repos.conf points to my local server, why would portage disregard that?

It doesn't.  If you point that at your local server, then that is
where the repo will come from, always.

One little caveat - if you change the setting and use git to sync,
then you need to either manually update your git repo settings (in
.git or using the command line), or delete the repo and let it resync.
Otherwise for some operations git's own saved location of the remote
uri will still be used.  And of course with git your mirror needs to
be a fast-forward of whatever it was syncing before if you don't wipe
it.  If you use rsync to sync then there is nothing to do here, as
rsync doesn't have any internal state like this or any concept of
history.

> >
> > The rsync server on the mirror host points to the gentoo portage installation on that local mirror host.  How can any metadata  there know about anything that's not already resolved there?

It can't.  That is why mirroring distfiles is more complex.  You need
to either mirror everything in the repo, or have some way to tell your
mirror host what all its downstream hosts actually need.

> > At the very least, I suspect that that local mirror page is wrong and rather refers to something that *could* be implemented (without extra packages being installed, just by configuration), but isn't yet.

Nope, just that there are two types of mirrors, and it tells you how
to do the "easy" one.  Distfiles are also easy to mirror but only if
you're willing to accept partial coverage.  Even partial coverage of
distfiles will likely save far more bandwidth than the repo mirror,
since the system set alone probably takes up a ton of space.

-- 
Rich