Gentoo Archives: gentoo-dev

From: Sebastian Pipping <webmaster@××××××××.org>
To: gentoo-dev@l.g.o
Cc: Zac Medico <zmedico@g.o>, mirror-admin@g.o
Subject: Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)
Date: Sun, 05 Jul 2009 13:52:45
Message-Id: 4A50B02A.7070207@hartwork.org
In Reply to: Re: [gentoo-dev] rsync mirrorstats page (generation and parsing) by "Robin H. Johnson"
1 Robin H. Johnson wrote:
2 > On Sun, Jul 05, 2009 at 02:44:07AM +0200, Sebastian Pipping wrote:
3 >> When collecting information on the SYNC variable for my Summer of Code
4 >> gentoo stats project I'd like to check if the URL in SYNC is publically
5 >> known or some private/secret rsync mirror. The page behind
6 >> http://mirrorstats.gentoo.org/rsync/
7 > Mirrorstats is known to be out of date, because somebody needs to sit
8 > down and integrate it with the datasources, so manual updates aren't
9 > needed. Even better, would be hooking it into bouncer2 for the sentry
10 > output.
11
12 What are these datasources?
13
14 What kind of integration are you thinking of?
15
16
17 > It needs somebody to update it and hook at into the SOURCE of this data:
18 > http://www.gentoo.org/main/en/mirrors3.xml
19 >
20 > But wait, you say, that page is distfiles mirrors? Mirror-admin have a
21 > common data source, non-published as it contains private contact details
22 > for each administrator. From that data source, mirrors3 and rsync
23 > mirrors gets updated.
24
25 I see.
26
27
28 > mirrors.xml - old page, only used by mirrorselect now, manually updated.
29 > mirrors3.xml - new page, generated from internal dataset.
30 > mirrors2.xml - not a real page (See
31 > http://www.gentoo.org/main/en/mirrors2.xml?passthru=1 and the magic
32 > <mirrorlist> element.
33
34 Compared to
35
36 [..]/mirrors.xml?passthru=1
37
38 it seems to me that on mirror3
39
40 [..]/mirrors3.xml?passthru=1
41
42 passthru= is working in the opposite direction:
43
44 1 turns style sheets on
45 0 turns them off (default)
46
47 The one for mirrors3 makes less sense to me.
48 Is this inconsistency intended?
49
50
51 > Relatedly, the original author of mirrorselect retired from Gentoo
52 > several years ago. The tools-portage team maintain it now, so you should
53 > co-operate with them. It would be nice if they implemented the mirrors3
54 > usage too, I think mirror-admin asked them more than a year ago, but I
55 > can't find the bug.
56
57 I agree that would be a good idea and another reason to touch
58 mirrorselect. Does it have a source repo somewhere?, not seen any.
59
60
61 > In the meantime, for your original question:
62 >> is the URL in SYNC public or private
63 > Simply check by matching against gentoo.org$ in the hostname part of the
64 > field.
65
66 Good idea, now implemented:
67
68 http://git.goodpoint.de/?p=smolt-gentoo.git;a=commitdiff;h=aeb14433e7c29a6045fb702775a3455ebb61aa1d
69
70
71 > P.S. Please report empty SYNC variables too ;-). These turn up when
72 > users/devs have their tree coming from a VCS instead of rsync.
73
74 Good point. Now also implemented, same commit as above.
75
76
77
78 Sebastian