Gentoo Archives: gentoo-dev-announce

From: "Robin H. Johnson" <robbat2@g.o>
To: gentoo-dev@l.g.o
Cc: gentoo-dev-announce@l.g.o, gentoo-core@l.g.o
Subject: [gentoo-dev-announce] Followup notes: {cvs,git,git.overlays}.gentoo.org migration; awol: some overlays commits, gitweb
Date: Mon, 18 Aug 2014 23:10:41
Message-Id: robbat2-20140818T220323-868929512Z@orbis-terrarum.net
In Reply to: [gentoo-dev-announce] cvs.gentoo.org, git.gentoo.org, *.overlays.gentoo.org migration timeline & ssh keys by "Robin H. Johnson"
1 Hi all,
2
3 Last evening, the old sponsor where cvs/git/git.overlays was hosted
4 turned off the old servers, earlier than I expected.
5
6 With two notable exceptions listed below, almost everything should be
7 how it should be, so you can continue as before.
8
9 The new SSH keys, in case you still didn't have them:
10 On Mon, Jun 30, 2014 at 10:26:52PM +0000, Robin H. Johnson wrote:
11 > 1024 5f:c3:fe:9a:ac:a7:99:f4:d3:c1:93:4c:52:87:74:28 (DSA)
12 > 256 aa:6a:e4:74:1d:73:d2:5a:9f:45:9f:18:55:81:c9:9a (ECDSA)
13 > 256 1c:2e:99:7d:c7:f0:bc:3b:a9:fb:d0:3e:2c:2a:79:ba (ED25519)
14 > 2048 24:3b:2d:3b:47:ca:7e:62:48:97:49:6a:f5:ad:66:88 (RSA)
15
16 1.
17 Overlays missing commits since 2014/07/22 02:49 UTC
18 ---------------------------------------------------
19 So last time overlays had an outage, infra promised to look into a git
20 mirroring setup, so when this forced migration was announced, I started
21 to implement a very rough version of mirroring for both the main cvs/git
22 repos, as well as the overlays repos.
23
24 It worked on the primary repos, but after some initial runs it failed on
25 all of the git.overlays repos at Jul 22 02:49 UTC. I've been busy with
26 real life, and I unfortunately didn't notice it until too late.
27
28 As a direct result, any commits pushed to git.overlays repos after
29 2014/07/22 22 02:49 UTC, are missing. If you have them locally, all you
30 need to do is 'git push' for all of your branches, and they'll be
31 restored.
32
33 2.
34 gitweb/cgit offline, anongit rate-limited
35 -----------------------------------------
36 One of the plans was to merge the hosting of git.gentoo.org and
37 git.overlays.gentoo.org into a single large Gitolite deployment, in
38 preparation for the Git migration of gentoo-x86.
39
40 The gotcha is that while we want all of the write traffic to be
41 consolidated, we want to be able to scale the readonly capacity easily
42 (without the budget or sharding that GitHub has).
43
44 It isn't a 100% solved problem yet (one of the reasons that I hadn't
45 flipped stuff over yet). There is also a lot more read-only Git traffic
46 than I expected (enough to kill some systems); so I'm NOT enabling a web
47 server on the box that runs the readwrite traffic yet.
48
49 The leading solution I have, that allows the most flexibility, is simply
50 splitting at the hostname:
51 http://gitweb.gentoo.org/$REPO
52 git://readonly.git.gentoo.org/$REPO
53 git://ro.git.gentoo.org/$REPO
54 git+ssh://git@××××××××××××××××××××.org/$REPO
55 git+ssh://git@×××××××××××××.org/$REPO
56
57 This also lets us trivially distribute readonly mirrors distributed
58 around the world, which would hugely improve performance when the Git
59 migration is done.
60
61 As a migration plan, git.gentoo.org would continue to host R+RW git for
62 a month.
63
64 Any concerns or alternative ideas should be proposed on the gentoo-dev
65 lists.
66
67 --
68 Robin Hugh Johnson
69 Gentoo Linux: Developer, Infrastructure Lead
70 E-Mail : robbat2@g.o
71 GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85