1 |
On 10/18/19 12:28 AM, Michał Górny wrote: |
2 |
> Rewrite the layout.conf getter to reuse the standard fetch() method |
3 |
> rather than using urlopen(). While at it, fix negative cache elision |
4 |
> to apply to memory cache as well (and not get written to disk if next |
5 |
> mirror was fine). |
6 |
> |
7 |
> Most importantly, this ensures that we respect FETCHCOMMAND while |
8 |
> fetching layout.conf, and so layout.conf is fetched the same way normal |
9 |
> distfiles are. With some uncommon configurations, the previous disjoint |
10 |
> logic might have resulted in one of the fetches failing while the other |
11 |
> succeeded. |
12 |
> |
13 |
> This also adds some nice verbosity. If mirror connection takes a while, |
14 |
> the user sees that rather than having Portage wait silently. |
15 |
> |
16 |
> Bug: https://bugs.gentoo.org/697566 |
17 |
> Signed-off-by: Michał Górny <mgorny@g.o> |
18 |
> --- |
19 |
> lib/portage/package/ebuild/fetch.py | 36 +++++++++++++++-------------- |
20 |
> 1 file changed, 19 insertions(+), 17 deletions(-) |
21 |
> |
22 |
> diff --git a/lib/portage/package/ebuild/fetch.py b/lib/portage/package/ebuild/fetch.py |
23 |
> index debab38a2..599c496ee 100644 |
24 |
> --- a/lib/portage/package/ebuild/fetch.py |
25 |
> +++ b/lib/portage/package/ebuild/fetch.py |
26 |
> @@ -382,7 +382,7 @@ class MirrorLayoutConfig(object): |
27 |
> return ret |
28 |
> |
29 |
> |
30 |
> -def get_mirror_url(mirror_url, filename, cache_path=None): |
31 |
> +def get_mirror_url(mirror_url, filename, mysettings, cache_path=None): |
32 |
> """ |
33 |
> Get correct fetch URL for a given file, accounting for mirror |
34 |
> layout configuration. |
35 |
> @@ -408,23 +408,25 @@ def get_mirror_url(mirror_url, filename, cache_path=None): |
36 |
> if ts >= time.time() - 86400: |
37 |
> mirror_conf.deserialize(data) |
38 |
> else: |
39 |
> + tmpfile = '.layout.conf.%d' % time.time() |
40 |
|
41 |
We'd need something more like mkstemp for real safety here. However, an |
42 |
alternative is to use a constant path (containing the mirror name), and |
43 |
assume that the fetch function updates files atomically. We can avoid |
44 |
interference with concurrent processes if we let files remain in DISTDIR |
45 |
(rather than unlink them). |
46 |
|
47 |
> try: |
48 |
> + if fetch({tmpfile: (mirror_url + '/distfiles/layout.conf',)}, |
49 |
> + mysettings, fetchonly=1, try_mirrors=0): |
50 |
> + tmpfile = os.path.join(mysettings['DISTDIR'], tmpfile) |
51 |
> + try: |
52 |
> + mirror_conf.read_from_file(tmpfile) |
53 |
> + finally: |
54 |
> + os.unlink(tmpfile) |
55 |
> + else: |
56 |
> + raise IOError() |
57 |
-- |
58 |
Thanks, |
59 |
Zac |