Re: [gentoo-alt] gemato in prefix

[Fabian Groffen]

Hi Michael,

To give you an idea; the current tree is getting its Manifests from
hashgen.c, which you can find in scripts/rsync-generation/hashgen.c.
The hashverify tool, which I'm currently working on, is basically an
addition to that file (doing argv[0] detection) to perform the
verification.  At this time of writing, I have the gpg-verification and
single file entry verification in place.  I'm still trying to close the
gap in checking the dirs in particular looking for files that are not
listed in the Manifest.
hashgen currently runs in 30s or so on the tree to generate manifests.
I hope it can verify in the same amount of time (we're talking about a
Quad G5 PowerPC machine here with rusty old spinning disks), leaving it
in a much better position to be used for Prefix, since we tend to have
slower/older machines around.  We're not really looking forward to 15
minutes of verification as some bugs have been reported to with gemato.

Portage used to do 1) checking the digests of ebuilds and 2) checking
for missing and extra files.  I noticed that at least 1) is no longer
present, which I find weird.  I need checking this on normal Gentoo,
(simply edit an ebuild and try to emerge it without updating its
digest), but I have the suspicion this got disabled because the full
tree verification should catch this.  Needless to say, that's
suboptimal, and not very secure IMO.

Now I may be all wrong with trying to implement the verification myself,
but that's a separate topic.  gemato should work fine.

I've checked some of the digests you mentioned and they look ok.  So I'm
wondering whether perhaps you got caught in the middle of a sync.  This
used to be much less of a problem because of per-ebuild-dir integrity,
but now the entire tree requires to be consistent.  I'll look into
re-activating my symlink-flip, which should make the switch atomic, but
I don't know what rsync is doing if the symlink is flipped during a
sync.  It reduces the invalid window somewhat I guess.

Thanks,
Fabian


On 20-02-2018 21:24:03 +0100, Michael Weiser wrote:
> Hi Fabian,
>
> On Tue, Feb 20, 2018 at 08:41:57PM +0100, Fabian Groffen wrote:
>
> > Thing is I once believed Portage checked manifest and all, but it seems
> > not to do anything any more, so my idea of things being OK may have been
>
> I also was a bit surprised to find that portage didn't authenticate and
> verify the tree at all. Stumbling over webrsync more or less by
> accident, I've been using it as the next best thing in the interim.
>
> From what I was able to find on the net, there's never been any
> actual implementation before Michal Gorny started gemato (see
> https://www.gentoo.org/glep/glep-0074.html#motivation wrt GLEP-58 from
> 2008 never being implemented). After using gemato on Gentoo Linux as a
> very early adopter I'm eager to get something comparable going in Prefix
> Mac.
>
> How can I help?
> --
> Thanks,
> Michael
>

-- 
Fabian Groffen
Gentoo on a different level