Re: [gentoo-user] Re: Re[4]: Re: Portage, git and shallow cloning

[Rich Freeman]

On Sat, Jul 7, 2018 at 5:29 PM Martin Vaeth <martin@×××××.de> wrote:
>
> Rich Freeman <rich0@g.o> wrote:
> > On Sat, Jul 7, 2018 at 1:34 AM Martin Vaeth <martin@×××××.de> wrote:
> >>
> >> Biggest issue is that git signature happens by the developer who
> >> last commited which means that in practice you need dozens/hundreds
> >> of keys.
> >
> > This is untrue. [...]
> > It will, of course, not work on the regular git repo [...]
> > You need to use a repo that is signed by infra
> > (which typically includes metadata/etc as well).
>
> I was speaking about gentoo's git repository, of course
> (the one which was attacked on github), not about a Frankensteined one
> with metadata history filling megabytes of disk space unnecessarily.
> Who has that much disk space to waste?

Doesn't portage create that metadata anyway when you run it, negating
any space savings at the cost of CPU to regenerate the cache?

>
> For the official git repository your assertions are simply false,
> as you apprently admit: It is currently not possible to use the
> official git repo (or the github clone of it which was attacked)
> in a secure manner.
>

Sure, but this also doesn't support signature verification at all (at
least not by portage - git can of course manually verify any commit),
so your points still don't apply.

> > and as a bonus they want them prepended to
> > instead of appended so that rsync resends the whole thing instead of
> > just the tail...
>
> Your implicit claim is untrue. rsync - as used by portage - always
> transfers whole files, only.

rsync is capable of transferring partial files.  I can't vouch for how
portage is using it, but both the rsync command line program and
librsync can do partial file transfers.  However, this is based on
offsets from the start of the file, so appending to a file will result
in the first part of the file being identical, but prepending will
break rsync's algorithm.

>
> > But, this was endlessly debated before the decision was made.
>
> The decision was about removing the ChangeLogs from the git
> repository. This was certainly the correct decision, because -
> as you said - the ChangeLogs *can* be regenerated from the
> git history and thus it makes no sense to modify/store them
> redundantly.

There were two decisions:

https://projects.gentoo.org/council/meeting-logs/20141014-summary.txt

"do we need to continue to create new ChangeLog entries once we're
operating in git?"  No.

https://projects.gentoo.org/council/meeting-logs/20160410-summary.txt

"The council does not require that ChangeLogs be generated or
  distributed through the rsync system. It is at the discretion of our
  infrastructure team whether or not this service continues."
  Accepted (4 yes, 1 no, 2 abstention)

> > It probably should be a configurable option in repos.conf, but
> > honestly, forced pushes are not something that should be considered a
> > good practice.
>
> 1. portage shouldn't decide about practices of overlays.

Hence the reason I suggested it should be a repos.conf option.

> 2. also in the official gentoo repository force pushes happen
>    occassionally. Last occurrence was e.g. when undoing the
>    malevolent forced push ;)

Sure, but that was a fast-forward from the last good commit, so it
wouldn't require a force pull unless a user had done a force pull on
the bad repo.

> 3. git pull fails not only for forced pushes but also in several
>    other occassions; for instance, if your filesystem changed inodes
>    numbers (e.g. squash + overlayfs after a resquash+remount).

If you're using squashfs git pull probably isn't the right solution for you.

> 4. Even if the user made the mistake to edit a file, portage should
>    not just die on syncing.

emerge --sync won't die in a situation like in general.  Maybe it will
if there is a merge conflict, but I don't think the correct default in
this case should be to just wipe out the user's changes.  I'm all for
making that an option, however.

-- 
Rich