Gentoo Archives: gentoo-user

From: Mick <michaelkintzios@×××××.com>
To: gentoo-user@l.g.o
Subject: Re: Aw: Re: Re: [gentoo-user] local mirrors (summary, leading to more questions)
Date: Tue, 30 Apr 2019 10:35:05
Message-Id: 13424297.NI6mnnaTES@dell_xps
In Reply to: Aw: Re: Re: [gentoo-user] local mirrors (summary, leading to more questions) by n952162@web.de
1 On Monday, 29 April 2019 22:50:18 BST n952162@×××.de wrote:
2 > But that seems strange - why would I need both GENTOO_MIRRORS and
3 > http_proxy?
4 >
5 > GENTOO_MIRRORS="http://myserver"
6 > http_proxy="http://myserver:3142
7 >
8 > Does the http_proxy imply that I'd need a proxy app, like squid. Between my
9 > client and server, there won't be any appreciable traffic.
10
11 I think there are two different concepts here, which have been conflated. You
12 do not need a proxy server, unless you intend to set up a caching server.
13
14 There are two configurations you could set up in your home network, depending
15 what functionality you intend to achieve:
16
17 1. Local portage and local distfiles servers.
18
19 You can set up an rsync server for portage's files and a web or ftp server for
20 distfiles. This works well with identical systems. When your local server
21 uses the same packages as your clients the corresponding distfiles will only
22 be downloaded once from the Gentoo source mirrors to the local server and all
23 clients will fetch them via web or ftp within your LAN, as and when they need
24 them.
25
26 However, when some of your clients have different packages installed, or in
27 any case different than the local server, then each client will need to
28 download such source files from the Internet. Say you have a server with no
29 Gnome packages, but 10 out of 30 client PCs run a Gnome desktop. Then each
30 and every of these 10 clients will try to fetch the Gnome distfiles from the
31 local server, get a 404 and then try the next server you have specified for
32 GENTOO_MIRRORS in their /etc/portage/make.conf. This way you end up
33 downloading large source files multiple times, which defeats the purpose of
34 running a local mirror. So, this set up works best for few LAN clients which
35 have mostly the same packages installed as the local Gentoo server and quickly
36 becomes suboptimal as the number of different packages x clients increases.
37
38 A variation on the above is using rsync to duplicate the distfiles directory
39 between server and client, or NFS/CIFS to share the same folder across PCs.
40
41
42 2. Local portage and local caching distfiles servers.
43
44 In this configuration the portage files are served over rsync as above. The
45 distfiles however are served via a caching proxy server. Using the previous
46 example, when some clients want to install different packages they will always
47 go via the local http_proxy server. If the server has cached said packages it
48 will serve these to the requesting clients immediately. Otherwise, the server
49 will fetch the missing packages from the Gentoo mirrors once only, cache them
50 locally and then serve these to whichever client requested them.
51
52
53 There are many different web servers you could use to serve distfiles. I have
54 used boa for many years because the server machine did not have a web/ftp
55 server already installed and the boa web server is exceedingly fast with a
56 very small footprint. However, if you already have apache running, then it
57 would make sense you use that.
58
59 There are also many proxy caching servers you could choose. Squid has already
60 been mentioned, net-misc/apt-cacher-ng is also mentioned in the wiki pages.
61
62 HTH.
63 --
64 Regards,
65 Mick

Attachments

File name MIME type
signature.asc application/pgp-signature