1 |
On 10/18/2019 09:41, Michał Górny wrote: |
2 |
> Hi, everybody. |
3 |
> |
4 |
> It is my pleasure to announce that yesterday (EU) evening we've switched |
5 |
> to a new distfile mirror layout. Users will be switching to the new |
6 |
> layout either as they upgrade Portage to 2.3.77 or -- if they upgraded |
7 |
> already -- as their caches expire (24hrs). |
8 |
> |
9 |
> The new layout is mostly a bow towards mirror admins, for some of whom |
10 |
> having a 60000+ files in a single directory have been a problem. |
11 |
> However, I suppose some of you also found e.g. the directory index |
12 |
> hardly usable due to its size. |
13 |
> |
14 |
> Throughout a transitional period (whose exact length hasn't been decided |
15 |
> yet), both layouts will be available. Afterwards, the old layout will |
16 |
> be removed from mirrors. This has a few implications: |
17 |
> |
18 |
> 1. Users who don't upgrade their package managers in time will lose |
19 |
> the ability of fetching from Gentoo mirrors. This shouldn't be that |
20 |
> much of a problem given that the core software needed to upgrade Portage |
21 |
> should all have reliable upstream SRC_URIs. |
22 |
> |
23 |
> 2. mirror://gentoo/file URIs will stop working. While technically you |
24 |
> could use mirror://gentoo/XX/file, I'd rather recommend finally |
25 |
> discarding its usage and moving distfiles to devspace. |
26 |
> |
27 |
> 3. Directly fetching files from distfiles.gentoo.org will become |
28 |
> a little harder. To fetch a distfile named 'foo-1.tar.gz', you'd have |
29 |
> to use something like: |
30 |
> |
31 |
> $ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2 |
32 |
> 1b |
33 |
> $ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz |
34 |
> ... |
35 |
> |
36 |
> |
37 |
> Alternatively, you can: |
38 |
> |
39 |
> $ wget http://distfiles.gentoo.org/distfiles/INDEX |
40 |
> |
41 |
> and grep for the right path there. This INDEX is also a more |
42 |
> lightweight alternative to HTML indexes generated by the servers. |
43 |
> |
44 |
> |
45 |
> If you're interested in more background details and some plots, see [1]. |
46 |
> |
47 |
> [1] https://dev.gentoo.org/~mgorny/articles/improving-distfile-mirror-structure.html |
48 |
> |
49 |
|
50 |
So the answer I didn't really see directly stated here is, where do new |
51 |
distfiles need to go //now//? E.g., if on woodpecker, I currently cp a |
52 |
distfile to /space/distfiles-local. What is the new directory I need to |
53 |
use? And if mirror://gentoo/${FOO} is going away, for the new distfiles |
54 |
target, what would be the applicable prefix to use? |
55 |
|
56 |
Directly using devspace seems like a bad idea, IMHO. Once long ago, we all |
57 |
got chastised for doing exactly that. Too much possibility of fragmentation |
58 |
as devs retire or package maintainership changes hands. |
59 |
|
60 |
I looked at the whitepaper'ish-like writeup, and I kinda don't like using a |
61 |
hash-based naming scheme on the new distfiles layout. I really kind prefer |
62 |
breaking the directories up based on the first letter of the distfiles in |
63 |
question, factoring case-sensitivity in (so you'd have 52 top-level |
64 |
directories for A-Z and a-z, plus 10 more for 0-9). Under each of those |
65 |
directories, additional subdirectories for the next few letters (say, |
66 |
letters 2-3). Yes, this leads to some orphan cases where a distfile might |
67 |
live on its own, but from a direct navigation standpoint, it's easy to find |
68 |
for someone browsing the distfiles server and easy to predict where a |
69 |
distfile is at. |
70 |
|
71 |
No math, statistical analysis, or deep-rooted knowledge of filesystems |
72 |
behind that paragraph. Just a plain old unfiltered opinion. Sometimes, I |
73 |
need to go get a distfile off the Gentoo mirrors, and being able to quickly |
74 |
find it in the mirror root is great. Having to do hash calculations to work |
75 |
out the file path will be *really* annoying. |
76 |
|
77 |
-- |
78 |
Joshua Kinard |
79 |
Gentoo/MIPS |
80 |
kumba@g.o |
81 |
rsa6144/5C63F4E3F5C6C943 2015-04-27 |
82 |
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 |
83 |
|
84 |
"The past tempts us, the present confuses us, the future frightens us. And |
85 |
our lives slip away, moment by moment, lost in that vast, terrible in-between." |
86 |
|
87 |
--Emperor Turhan, Centauri Republic |