Re: [gentoo-dev] [RFC] New third party mirrors

[Corentin Chary]

On Thu, Apr 26, 2012 at 10:07 AM, Zac Medico <zmedico@g.o> wrote:
> On 04/26/2012 01:03 AM, Corentin Chary wrote:
>> On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico <zmedico@g.o> wrote:
>>> On 04/26/2012 12:30 AM, Corentin Chary wrote:
>>>> On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@g.o> wrote:
>>>>> On Wed, 25 Apr 2012 09:16:05 +0200
>>>>> Corentin Chary <corentin.chary@×××××.com> wrote:
>>>>>
>>>>>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@g.o>
>>>>>> wrote:
>>>>>>> On Tue, 24 Apr 2012 16:19:11 +0000
>>>>>>> "Robin H. Johnson" <robbat2@g.o> wrote:
>>>>>>>
>>>>>>>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>>>>>>>>>>> $ ./mirrors.py --all --count
>>>>>>>>>>> 297 ?? ?? http://pear.php.net
>>>>>>>>>>> 297 ?? ?? http://pear.php.net/get
>>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net
>>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net/get
>>>>>>>>>> These are already mirror bouncers. If you visit the above,
>>>>>>>>>> you'll get the closest mirror for downloading.
>>>>>>>>> And since there is already ~10 "mirrors" with only one actual
>>>>>>>>> backend, should they go to thirdpartymirrors or not ? If not,
>>>>>>>>> what about this pseudo-mirrors already present in
>>>>>>>>> thirdpartymirrors ?
>>>>>>>> I think we should add the pseudo-mirrors, but explicitly mark them
>>>>>>>> as such in the file, so that they don't get duplicate entries
>>>>>>>> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>>>>>>>> Should have just the bouncer).
>>>>>>>
>>>>>>> It'd be great if we could add some kind of additional mirror
>>>>>>> entries, which would be used by repoman to signal missing mirror://
>>>>>>> entries but won't be used for downloads.
>>>>>>
>>>>>> Yep, we could put that in it too:
>>>>>> github                http://github.com/downloads/
>>>>>> https://github.com/downloads/
>>>>>
>>>>> Per spec, portage can choose a random mirror of the list. If we put
>>>>> entries like that, these two will be equally possible as the preferred
>>>>> cloud. URL -- while they redirect one to another.
>>>>>
>>>>> We might decide on some common syntax like preceding all extra entries
>>>>> with '-' but I don't want to be the one deciding here.
>>>>
>>>> I checked, and current portage code already handle entries starting
>>>> with a - gracefully thanks to stack_dictlist (removing them from the
>>>> list of mirrors).
>>>
>>> That means repoman will ignore them too. If you want existing versions
>>> of repoman to check for those paths in SRC_URI, you can add a line like
>>> this to thirdpartymirrors:
>>>
>>> github-bad-urls http://github.com/downloads/ https://github.com/downloads/
>>
>> Hum, I checked repoman source code, and I didn't find where it checks
>> if SRC_URI matches something in thirdpartymirror. Any hint ?
>
> Search for SRC_URI.mirror in /usr/bin/repoman.

Arg.. ok, I only looked in pym/repoman/.

So two solutions here:

First one:
github http://cloud.github.com/downloads -http://github.com/downloads/
-https://github.com/downloads/
+ a small patch that would allow repoman to do something like
settings.thirdpartymirrors(keep_bad_uris=True) in order to keep uris
starting with a - in the list.

Second solution:
github http://cloud.github.com/downloads
github-bad-uris -http://github.com/downloads/ -https://github.com/downloads/

The good thing with the first one is that it would allow repoman to
outputs something like "you should use 'mirror://github'".


-- 
Corentin Chary
http://xf.iksaif.net