Gentoo Archives: gentoo-dev

From: Kurt Lieber <klieber@g.o>
To: gentoo-dev@g.o
Subject: [gentoo-dev] The release of 1.4 and its impact on our mirrors
Date: Wed, 23 Jul 2003 19:50:53
Message-Id: 20030723194823.GJ9959@mail.lieber.org
1 Folks --
2
3 One issue that came out of today's manager meeting was the amount of space
4 that GRP and some of the new livecds will require for 1.4_release. As I
5 understand it, that number hovers right around 10GB at the moment.
6
7 We have already received numerous complaints from our mirror admins about
8 the amount of disk space we chew up now. For reference, here is a break
9 down:
10
11 9.7G ./releases
12 139M ./snapshots
13 17G ./distfiles
14 6.6G ./experimental
15
16 We can clean out some space in the /releases directory by deleting some of
17 the old 1.4_rc* stuff, but we will still have a significant overall size
18 increase as a part of 1.4_final. We will be working on reducing the size
19 of /distfiles as well, but this will be nothing more than a temporary,
20 tactical fix. We still need a longer term strategic fix.
21
22 I'd like to come up with a solution that allows each arch the flexibility
23 it requires to create new livecds, GRP packages, etc. But I'd also like to
24 be mindful and respectful of our mirrors.
25
26 I'd like to hear suggestions from you all on the best way to achieve this.
27 Some ideas that I've heard so far include:
28
29 * creating separate directories that are optional for our mirrors to
30 support. GRP would be a prime candidate for this given the space it
31 requires. Mirrors short on disk space could choose not to mirror these
32 files.
33 * assigning each arch a quota for the files it may use outside of
34 /distfiles. This quota would apply to liveCDs, things in /experimental
35 as well as everything within /releases (as well as /grp if we make that a
36 separate top level directory) To be effective, this quota would need to
37 be around the ~3GB level given the number of arches we support.
38
39 Those are simply two ideas that have been proposed so far. I'm sure you
40 guys will have more. The only requirements at this point are:
41
42 * We cannot expect our system of mirrors to give us an unlimited amount of
43 disk space. (they've already started to complain)
44 * We cannot afford to lose many mirrors. Especially in North America,
45 source mirrors are in short supply.
46 * We must expect /distfiles to continue growing as we continue to add new
47 ebuilds. So solutions involving "reducing the size of /distfiles" will
48 not be feasible except as a short-term, stop-gap measure.
49
50 Thoughts? Ideas?
51
52 --kurt

Replies

Subject Author
Re: [gentoo-dev] The release of 1.4 and its impact on our mirrors Matthew Walker <mwalker@×××××××.net>
Re: [gentoo-dev] The release of 1.4 and its impact on our mirrors Tal Peer <coredumb@g.o>
Re: [gentoo-dev] The release of 1.4 and its impact on our mirrors fede2@fuerzag.ulatina.ac.cr (Alvaro Figueroa Cabezas)
[gentoo-dev] (crazy?) proposal to reduce load and disk on mirrors "Håvard Wall" <haavardw@×××××××.no>