Gentoo Archives: gentoo-dev

From: Roy Bamford <neddyseagoon@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure
Date: Sat, 27 Jan 2018 11:37:03
Message-Id: o05VdtdvSqu5k1NsFIf8Bs@mEcF0NoJxoIPixgXzZEMs
In Reply to: Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure by "Michał Górny"
1 On 2018.01.27 08:30, Michał Górny wrote:
2 > W dniu pią, 26.01.2018 o godzinie 20∶48 -0500, użytkownik Michael
3 > Orlitzky napisał:
4 > > On 01/26/2018 06:24 PM, Michał Górny wrote:
5 > > >
6 > > > The alternate option of using file hash has the advantage of
7 > having
8 > > > a more balanced split. Furthermore, since hashes are stored
9 > > > in Manifests using them is zero-cost. However, this solution has
10 > two
11 > > > significant disadvantages:
12 > > >
13 > > > 1. The hash values are unknown for newly-downloaded distfiles, so
14 > > > ``repoman`` (or an equivalent tool) would have to use a
15 > temporary
16 > > > directory before locating the file in appropriate subdirectory.
17 > > >
18 > > > 2. User-provided distfiles (e.g. for fetch-restricted packages)
19 > with
20 > > > hash mismatches would be placed in the wrong subdirectory,
21 > > > potentially causing confusing errors.
22 > > >
23 > >
24 > > The filename proposal sounds fine, so this is only academic, but:
25 > are
26 > > these two points really disadvantages?
27 > >
28 > > What are we worried about in using a temporary directory? Copying
29 > across
30 > > filesystem boundaries? Except in rare cases, $DISTDIR itself will be
31 > > usable a temporary location (on the same filesystem), won't it?
32 >
33 > Why add the extra complexity when there's no need for one? Note that
34 > there's also the problem of resuming transfers, so in the end we're
35 > talking about permanent temporary directory where we keep unfinished
36 > transfers.
37 >
38 > > For the second point, portage is going to tell me where to put the
39 > file,
40 > > isn't it? Then no matter what garbage I download, won't portage look
41 > for
42 > > it in the right place, because where-to-put-it is determined using
43 > the
44 > > same manifest hash that determines where-to-find-it?
45 >
46 > No, it won't. Why would it? You're going to call something like:
47 >
48 > edistadd foo.tar.gz bar.tar.gz
49 >
50 > ...and it will place the files in the right subdirectories.
51 >
52 > --
53 > Best regards,
54 > Michał Górny
55 >
56 >
57 >
58 >
59
60 Michał,
61
62 How does this work for fetch restricted files and finding other files
63 no longer on the mirrors?
64
65 Its no longer a download and move it to $DISTFILES, or is it?
66 Whatever it is, users will need to do it unless files in $DISTFILES
67 are accepted by package managers if they are not found in the main
68 structure.
69
70 --
71 Regards,
72
73 Roy Bamford
74 (Neddyseagoon) a member of
75 elections
76 gentoo-ops
77 forum-mods

Replies

Subject Author
Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure "Michał Górny" <mgorny@g.o>