1 |
Ok, I've got my verification tool up and running on the main gx86 rsync |
2 |
tree now (5s on hot filesystem) now I'm seeing a lot of hash mismatches |
3 |
for the prefix tree. One step further I think. |
4 |
|
5 |
Fabian |
6 |
|
7 |
|
8 |
On 22-02-2018 20:52:33 +0100, Michael Weiser wrote: |
9 |
> Hi Fabian, |
10 |
> |
11 |
> On Thu, Feb 22, 2018 at 08:24:25AM +0100, Fabian Groffen wrote: |
12 |
> |
13 |
> > Please excuse my snipping. |
14 |
> |
15 |
> I was rambling, wasn't I. :) I'll snip some more for now. |
16 |
> |
17 |
> > > > hashgen currently runs in 30s or so on the tree to generate |
18 |
> > > > manifests. |
19 |
> I recompiled gcc overnight and got the following numbers today on one |
20 |
> of the ARM SBCs: |
21 |
> |
22 |
> [root@linux:~] time ./hashgen /usr/portage/ |
23 |
> real 7m28.071s |
24 |
> user 10m8.355s |
25 |
> sys 0m57.435s |
26 |
> [root@linux:~] time ./hashgen /usr/portage/ |
27 |
> real 7m6.195s |
28 |
> user 9m54.715s |
29 |
> sys 0m54.123s |
30 |
> |
31 |
> [root@linux:~] time gemato create -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg -f -S -t -H SHA512\ BLAKE2B /usr/portage |
32 |
> INFO:root:Creating Manifests in /usr/portage... |
33 |
> INFO:root:/usr/portage updated in 599.46 seconds |
34 |
> |
35 |
> real 10m5.431s |
36 |
> user 8m54.606s |
37 |
> sys 0m59.072s |
38 |
> |
39 |
> So there is a 30% speedup at least. |
40 |
> |
41 |
> > Odd, was this rsync1 or rsync2? |
42 |
> [...] |
43 |
> > path. Instead rsync1 and rsync2 are doing full generation both of them. |
44 |
> > To make this be equal, I had to play a trick with timestamps, I |
45 |
> > basically set the timestamp to the git commit time. Maybe this plays a |
46 |
> > role too? |
47 |
> |
48 |
> I just retried: timestamp.chk seemed to be in sync between rsync1 and |
49 |
> rsync2 at first: A sync from rsync1 syncs some files and later syncs |
50 |
> even against rsync2 find timestamp.chk matching. |
51 |
> |
52 |
> When I delete timestamp.chk and retry several times until the other |
53 |
> server round-robins in, emaint sync rsyncs what seems like all |
54 |
> second-level Manifest.gzs (net-fs, dev-libs, etc.) and what seems like |
55 |
> the whole metadata/md5-cache directory. |
56 |
> |
57 |
> When having synced from rsync2, gemato verify failed on |
58 |
> dev-cpp/Manifest.gz. When having synced from rsync1, it failed on |
59 |
> net-fs/Manifest.gz. |
60 |
> |
61 |
> After waiting for the next refresh, emaint synced selective 2nd level |
62 |
> manifests and md5-cache from rsync2 again. It still failed verification |
63 |
> on dev-cpp/Manifest.gz. |
64 |
> |
65 |
> On rsync1, timestamp.chk still differed after that first sync and emaint |
66 |
> sync again synced selective 2nd and 3rd level manifests and again what |
67 |
> seemed like the whole md5-cache. Verification with gemato still failed |
68 |
> on net-fs/Manifest.gz. |
69 |
> |
70 |
> So the hash discrepancies seem to survive regeneration. |
71 |
> |
72 |
> Then I deleted the whole $PORTDIR and sync from scratch. This was |
73 |
> against rsync1. Verification still fails on net-fs/Manifest.gz. |
74 |
> |
75 |
> Deleting timestamp.chk again and running against rsync2, it behaved as |
76 |
> before and again failed at dev-cpp/Manifest.gz. |
77 |
> |
78 |
> $ gemato verify -K /Users/michael/b/pubring.gpg -R -j1 |
79 |
> /usr/local/gentoo/usr/portage/ |
80 |
> INFO:root:Manifest timestamp: 2018-02-22 19:26:44 UTC |
81 |
> INFO:root:Valid OpenPGP signature found: |
82 |
> INFO:root:- primary key: 0204A8ABD003E57A9558850DBA08091EC6317B3C |
83 |
> INFO:root:- subkey: 0204A8ABD003E57A9558850DBA08091EC6317B3C |
84 |
> INFO:root:- timestamp: 2018-02-22 19:26:44 UTC |
85 |
> INFO:root:Verifying /usr/local/gentoo/usr/portage/... |
86 |
> ERROR:root:Manifest mismatch for dev-cpp/Manifest.gz |
87 |
> BLAKE2B: expected: |
88 |
> 64d7c4bd55a14e8c7296e8185ad9654db39cfc095107411c97b5f425856f780c0b2dffcef436c07bc07c8832506943e7d80ab5eaf2923eb4bc419dea3a8d071a, |
89 |
> have: |
90 |
> 6b698c9af8c1bf5012ee01ea308718b2f09330a181b48e663a27977885b75bd439a64d568d59de6a2a17bcad86cedf0cd8cda28361155c382badadc0d369843d |
91 |
> SHA512: expected: |
92 |
> a7d12f2653817a47cc76de6850f8a9ab22bb952f2df1d1029cb23805f868b1d6610a2bc35d1f13666890ed1c9648907e25e949c78f75e2318065b400872e719f, |
93 |
> have: |
94 |
> 070bb46740c8ecc565d23dcc35cedc3bb96d6014b083da8428be00e4008ef2e2d882d3b8e8047fe84e16542b090d02c881c08ce60b052de76406f65caf6dd893 |
95 |
> |
96 |
> Can you perhaps confirm these hash discrepancies on the servers? |
97 |
> |
98 |
> > I only checked rsync1 last time, maybe rsync2 isn't as equal as I |
99 |
> > thought. |
100 |
> |
101 |
> They both individually seem to produce hash discrepancies on different |
102 |
> files. |
103 |
> -- |
104 |
> Thanks, |
105 |
> Michael |
106 |
> |
107 |
|
108 |
-- |
109 |
Fabian Groffen |
110 |
Gentoo on a different level |