Re: [gentoo-project] [rfc] /usr/portage sync source options (and recommendations)

[Brian Dolbec]

On Wed, 28 Dec 2016 02:37:53 +0100
Sebastian Pipping <sping@g.o> wrote:

> Hi there!
>
>
> My impression is that:
>
>   * a large number of people still sync /usr/portage using plain
> rsync,
>
>   * it is not a trivial choice which source to switch to.
>
> Do we have documentation comparing options for syncing /usr/portage?
>
>
> The rest of this mail is a collection of methods that I am aware of.
> It could become the start of a wiki page if missing or so.
>
> Best
>
>
>
> Sebastian
>
>
> ===
>
> Sync options that I am aware of include:
>
>   * Plain rsync
>
>   * Git: 4x direct source (Ebuilds + DTD + GLSA + News)
>
>   * Git: combined mirror [1]
>
>   * webrsync: Plain
>
>   * webrsync: With GPG [2]
>
>   * ..?
>
>
> The key questions for differences and picking I see are:
>
>   * Does it use secure transport, e.g. SSL?

The webrsync module needs to be re-written in python to better fit the
new plugin-sync system.  It can then take advantage of gkeys use of the
ssl-fetch lib which layman and mirrorselect are using for verified ssl
connections and downloads, complete with gpg verification.  Zac made a
patch for the current webrsync module that uses gkeys for verification,
but it is waiting for a stable release of gkeys before it will be
released.

>
>   * Does it verify GPG signatures?

Currently, Only GPG enabled webrsync.  The meta-manifest system code
from gsoc-2016 is in the process of being worked on and will be merged
into portage very soon.  It needs a couple updates for the Changelog
changes (I believe) and I am working to get another gkeys release out
that should be able to go stable.  With that the meta-manifest code and
gkeys use will go into portage, so the new system will be capable of
normal rsync with partial tree downloads.  It will then gpg verify
everything in the local tree.


>
>   * Does it support incremental updates?


A new meta-manifest enabled rsync tree will support both gpg
verification as well as partial tree downloads.

>
>   * Does it come with metadata/md5-cache pre-generated?
>
>   * How much delay does it have?
>
>
> [1] https://github.com/gentoo-mirror/gentoo
> [2]
> http://blog.siphos.be/2011/07/emerge-webrsync-and-gpg-verification/
>

I hope to get a gkeys release out by the end of the current holidays
which includes all teh gsoc-2016 work.  Plus some other changes that
should make gkeys stable (including a semi-automated update system for
users. With a little help, the gkeys-ldap portion of the repo will begin
to get merged into infra to auto-generate the seed files.

Also, infra had been looking at getting keycards for new infra
openpgp keys which will be used as part of the gpg process.

-- 
Brian Dolbec <dolsen>