Re: [gentoo-alt] gemato in prefix

[Michael Weiser]

Hi Fabian,

On Sun, Mar 04, 2018 at 09:20:31PM +0100, Fabian Groffen wrote:

> > Were you able to pin down the actual cause of the verification failures?
> Yes.
> (BRACE BRACE)

Now that's what I call a tough nut to crack. Awesome work to sort it
out.

> > gemato is down to 24 seconds for verification, hashverify now takes 13
> > seconds.
> Darn, and how many cpu cores does it use throughout?  Just two?

I think you can relax. :) After upgrade from 10.3 to 11.2 it now runs
more than a minute again. In four runs I've seen 66, 87, 56 and 72
seconds.

From the PIDs, process tree and CPU load it seems to start one
controller and four workers (the i7-5650U has Hyperthreading). Work
doesn't seem to be distributed equally between the workers because they
finish at different times. The first finishes after about twenty
seconds. The other follow in 10 second intervals. At the end the
controller seems to do some more work for another 10 seconds.

It gets slower with just two workers (-j2). For some reason I can't seem
to saturate the CPUs. It can't be I/O per se because right afterwards
hashverify goes up to 272% CPU usage and finishes in 8 to 11 seconds. So
the fs cache must be fully primed.

> > I'll go on and try to integrate gemato and hashverify with emaint sync
> > in prefix.
> Gemato should be already pretty much hooked, isn't it?  I assume for now
> we could use a hook?

It is in mainline portage. There it now lives as a hardcoded call to
gemato in rsync.py as seen with 2.3.22 on Linux. Prefix portage 2.3.18
seems to lack it.

Is it a lot of work to make a prefix portage release in line with
mainline portage?

I've installed the repos.postsync.d hook from the source distribution
for now and it seems to work fine. Same for hashverify. So I now have
extra-pranoia double-verification using two implementations. :)

How are chances to get the Prefix signing key into gentoo-keys?
-- 
thanks, Micha