Gentoo Archives: gentoo-dev

From: Fabian Groffen <grobian@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] New distfile mirror layout
Date: Tue, 29 Oct 2019 14:33:14
Message-Id: 20191029143304.GC22441@gentoo.org
In Reply to: Re: [gentoo-dev] New distfile mirror layout by Ulrich Mueller
1 On 29-10-2019 15:17:38 +0100, Ulrich Mueller wrote:
2 > >>>>> On Tue, 29 Oct 2019, Michał Górny wrote:
3 >
4 > > On Tue, 2019-10-29 at 14:09 +0100, Ulrich Mueller wrote:
5 > >> > What if the file is hosted at a non-standard tcp port upstream
6 > >> > (like http://example.org:8080/)? The devmanual says that it _must_
7 > >> > be manually uploaded to /space/distfiles-local/ in such cases.
8 >
9 > >> Or another example, app-emacs/vhdl-mode-3.38.1, where (incompetent,
10 > >> or nasty?) upstream blocks wget for some reason, but other methods
11 > >> (e.g., curl, firefox) work? How would I get the file onto the mirrors
12 > >> there?
13 >
14 > > If I were you, I would've explicitly mirrored the file anyway.
15 > > If upstream blocks wget, then users who do not use GENTOO_MIRRORS will
16 > > also suffer due to it.
17 >
18 > All what I'm saying is that there can be unusual circumstances where
19 > manual uploading of a file is useful. So please don't take that
20 > possibility away.
21
22 In addition, there are currently files there that aren't referenced from
23 ebuilds. Prefix uses these files during bootstrap, local mirrors are
24 often much faster than dev.g.o.
25
26 If the files don't get mirrored anymore, I guess I can create a dummy
27 ebuild that has the files in SRC_URI.
28
29 If the files get mirrored, but put in a subdir based on the filename
30 hash, the original query endpoint on distfiles.g.o changes, much like
31 the SRC_URI approach.
32
33 Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o
34 URL with subdir for most part I think, but it's sub-optimal from my
35 point of view. Calculating the hash is not always feasible due to the
36 lack of b2sum or other means. Hence my earlier request to have such
37 official translation service on Gentoo hardware.
38
39 (I just wrote a small wsgi script that calculates the hash and generates
40 the redirect from Python, served via uwsgi/nginx, but there should be
41 many ways to achieve the same goals, if and only if a blake2b
42 implementation were available for it.)
43
44 Thanks,
45 Fabian
46
47 --
48 Fabian Groffen
49 Gentoo on a different level

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies

Subject Author
Re: [gentoo-dev] New distfile mirror layout "Michał Górny" <mgorny@g.o>