Gentoo Archives: gentoo-dev

From: R0b0t1 <r030t1@×××××.com>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] Managing updates on many identical Gentoo systems
Date: Thu, 18 Jan 2018 23:00:51
Message-Id: CAAD4mYgT5iUKbA-tfpsqePHr+drngpU6tgvtKCpp1LmBJPE73A@mail.gmail.com
In Reply to: [gentoo-dev] Managing updates on many identical Gentoo systems by "Anthony G. Basile"
1 Hello,
2
3 There's been a lot of relevant points but I'm not sure how to stitch
4 together all of the posts so I will reply to the OP.
5
6 On Thu, Jan 18, 2018 at 5:46 AM, Anthony G. Basile <blueness@g.o> wrote:
7 > Hi everyone,
8 >
9 > I'm trying to design an update system for many identical Gentoo systems.
10 > Using a binhost is obvious, but there are still problems with this
11 > approach.
12 >
13
14 How are the hosts administered now? Which problems are you referring to?
15
16 It's my understanding that Debian/Fedora/etc. based datacenters have
17 all of their machines crunching dependencies individually. There may
18 be local mirrors, but it looks like people have not seen fit to
19 centralize dependency information.
20
21 > Unless there's some magic I don't know about (and this is why I'm
22 > sending this email) each machine still needs to have the portage tree
23 > installed locally (1.5 GB) or somehow mounted by a network filesystem
24 > (which is not practical if the machines are not on a local network).
25 > Furthermore, each machine would have to run emerge locally to do the
26 > calculation of what packages need updating.
27 >
28 > This procedure is redundant because each machine is housing the same
29 > data and doing the same dependence-tree calculation. It should be
30 > possible to do this calculation on a centralized binhost and simply
31 > communicate the update information to the remote machines. They would
32 > then only have to download the .tbz2's and install them, keeping a tidy
33 > /var/db/pkg. Thus they avoid having to house the portage tree and
34 > burning cpu cycles that just calculate redundant information.
35 >
36
37 Is it necessary to retain information that would be used to administer
38 the machines on the machines? You could treat your servers as embedded
39 appliances. The typical method used in that case is to push an entire
40 root to them at once. For safety's sake a separate system partition
41 can receive the system image; system data is kept elsewhere.
42
43 This would require some small customization by way of update code to
44 unpack the new system and modify bootloader entries. All configuration
45 of the system apart from that would have to be handled upstream when
46 the release was prepared, or stored on the data partition.
47
48 This is done for lots of devices (phones?) and works well even when
49 bandwidth is fairly limited.
50
51 Cheers,
52 R0b0t1