Re: [gentoo-dev] rsync mirrorstats page (generation and parsing)

[Robin H. Johnson]

On Sun, Jul 05, 2009 at 02:44:07AM +0200, Sebastian Pipping wrote:
> When collecting information on the SYNC variable for my Summer of Code
> gentoo stats project I'd like to check if the URL in SYNC is publically
> known or some private/secret rsync mirror.  The page behind
>   http://mirrorstats.gentoo.org/rsync/
Mirrorstats is known to be out of date, because somebody needs to sit
down and integrate it with the datasources, so manual updates aren't
needed. Even better, would be hooking it into bouncer2 for the sentry
output.

It needs somebody to update it and hook at into the SOURCE of this data:
http://www.gentoo.org/main/en/mirrors3.xml

But wait, you say, that page is distfiles mirrors? Mirror-admin have a
common data source, non-published as it contains private contact details
for each administrator. From that data source, mirrors3 and rsync
mirrors gets updated.

mirrors.xml - old page, only used by mirrorselect now, manually updated.
mirrors3.xml - new page, generated from internal dataset.
mirrors2.xml - not a real page (See
http://www.gentoo.org/main/en/mirrors2.xml?passthru=1 and the magic
<mirrorlist> element.

Relatedly, the original author of mirrorselect retired from Gentoo
several years ago. The tools-portage team maintain it now, so you should
co-operate with them. It would be nice if they implemented the mirrors3
usage too, I think mirror-admin asked them more than a year ago, but I
can't find the bug.

In the meantime, for your original question:
> is the URL in SYNC public or private
Simply check by matching against gentoo.org$ in the hostname part of the
field.

P.S. Please report empty SYNC variables too ;-). These turn up when
users/devs have their tree coming from a VCS instead of rsync.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85