| 1 |
> On Oct 18, 2019, at 9:42 AM, Michał Górny <mgorny@g.o> wrote: |
| 2 |
> |
| 3 |
> Hi, everybody. |
| 4 |
> |
| 5 |
> It is my pleasure to announce that yesterday (EU) evening we've switched |
| 6 |
> to a new distfile mirror layout. Users will be switching to the new |
| 7 |
> layout either as they upgrade Portage to 2.3.77 or -- if they upgraded |
| 8 |
> already -- as their caches expire (24hrs). |
| 9 |
> |
| 10 |
> The new layout is mostly a bow towards mirror admins, for some of whom |
| 11 |
> having a 60000+ files in a single directory have been a problem. |
| 12 |
> However, I suppose some of you also found e.g. the directory index |
| 13 |
> hardly usable due to its size. |
| 14 |
This sounds like a filesystem issue. Do we know which filesystems are suffering? |
| 15 |
|
| 16 |
ZFS should be fine. I believe ext2/ext3 have problems with this many files. ext4 is probably okay, but don’t quote me on that. |
| 17 |
> |
| 18 |
> Throughout a transitional period (whose exact length hasn't been decided |
| 19 |
> yet), both layouts will be available. Afterwards, the old layout will |
| 20 |
> be removed from mirrors. This has a few implications: |
| 21 |
> |
| 22 |
> 1. Users who don't upgrade their package managers in time will lose |
| 23 |
> the ability of fetching from Gentoo mirrors. This shouldn't be that |
| 24 |
> much of a problem given that the core software needed to upgrade Portage |
| 25 |
> should all have reliable upstream SRC_URIs. |
| 26 |
> |
| 27 |
> 2. mirror://gentoo/file URIs will stop working. While technically you |
| 28 |
> could use mirror://gentoo/XX/file, I'd rather recommend finally |
| 29 |
> discarding its usage and moving distfiles to devspace. |
| 30 |
> |
| 31 |
> 3. Directly fetching files from distfiles.gentoo.org will become |
| 32 |
> a little harder. To fetch a distfile named 'foo-1.tar.gz', you'd have |
| 33 |
> to use something like: |
| 34 |
> |
| 35 |
> $ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2 |
| 36 |
> 1b |
| 37 |
> $ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz |
| 38 |
> ... |
| 39 |
> |
| 40 |
> |
| 41 |
> Alternatively, you can: |
| 42 |
> |
| 43 |
> $ wget http://distfiles.gentoo.org/distfiles/INDEX |
| 44 |
> |
| 45 |
> and grep for the right path there. This INDEX is also a more |
| 46 |
> lightweight alternative to HTML indexes generated by the servers. |
| 47 |
> |
| 48 |
> |
| 49 |
> If you're interested in more background details and some plots, see [1]. |
| 50 |
> |
| 51 |
> [1] https://dev.gentoo.org/~mgorny/articles/improving-distfile-mirror-structure.html |
| 52 |
> |
| 53 |
> -- |
| 54 |
> Best regards, |
| 55 |
> Michał Górny |
| 56 |
> |
Archives