Re: [gentoo-server] Wrapping my brain around a gold server.

[Andrew Cowie]

On Tue, 2003-08-05 at 03:29, Dormando wrote: 
> Currently I help manage a melange (over 300) of servers ranging from
> sunos, to redhat 6.2, to redhat 7.2+. I hope that mayhaps in a year or so
> the gentoo server project will be mature enough to start rolling out
> servers under infrastructure planning, and slowly rebuild old servers
> into the new setup. I'm relatively new here, and there's a hell of a lot
> of legacy in this network.

There always is legacy.

That fact has been a touch absent in these discussions. If we're really
smart, we won't just create a gentoo-gold-server which is only suitable
for use in networks of gentoo machines, we'll create a master server
which runs gentoo and which is suitable to act as gold server for a
whole suite of machines, regardless of OS.

Obviously, it'll be central to managing the emerge side of downstream
gentoo machines, but don't forget the bootstrapping paper's point: there
are many architectures in any environment, and you need to be able to
work with all of them.

Andrea's Barisani's email (which I forwarded) about having had to
organize a master server with directories for different processor
architectures probably needs to be extended with two other ideas:
classes of machines (as you're talking about, Dormando) and operating
systems.

You need:

Architecture
	P3, Athalon, UltraSPARC III (sun4u whatever), PowerPC

Operating System
	Gentoo Linux, Red Hat Linux, Debian Linux, Solaris Unix,
	AIX Unix, *BSD, [and yes, potentially even Mac OS X
	ultimately Windows] - whatever you have that, legacy or
	otherwise, you have to sustain

Class
	webserver, mailserver, MySql database, Postgres database,
	Oracle database, fileserver, authentication server, directory
	server, ...

Building *everything* (in the case of Gentoo or BSD) and/or mirroring
everything (in the case of Debian, RedHat, and the commercial Unixes)
for every permutation would be silly. But one idea that might be good
would be build (mirror) on demand.

To digress quickly, if you've ever worked with PHP, you know that you
can go to php.net/function and get a lookup of that function in the PHP
online manual. Every possible jump hasn't been figured out; rather none
of them are - they just use a 404 handler which has some intelligence to
find in the manual what you might be looking for. Yahoo uses the same
trick with the stock quote charts (eg
http://finance.yahoo.com/q?s=AUDUSD=X&d=c&t=my ). Every chart isn't
regenerated every moment; rather, if a request comes in for a chart the
cache is checked, and then if the request is for a chart that hasn't
been generated yet, then it gets generated, served, and added to the
cache. Forcing updates are a simple matter of wiping files from the
cache every so often.

The same idea might be useful with our gold server. Rather than
building/mirroring *every possible combination* ahead of time, we might
instead write things so that when they're requested the first time, we
go ahead and fetch/build it, and then let it be pulled down.

This is better, for example, than building mysql 13 times, once for each
OS/Architecture combination you have, when you only need it once or
twice. And when, next year, you need it for a new third combination,
that's when it gets prepared, ready to order).

You could, of course always use some simple tool to pre-load the cache
on your gold server (or, if you will, order up a certain combination) if
you know in advance you're going to need it.

[Someone mentioned network aware portage. Maybe, maybe not. But this
approach above might imply some sort of network level "world" style
file, although that's more a gold server concept than a Gentoo one]

For some related thoughts, you might want to see apt-cacher in Debian by
Nick Andrew and Jon Oxer: http://www.apt-cacher.org

AfC

-- 
Andrew Frederick Cowie
Operational Dynamics Consulting Pty Ltd

Australia [phone]  North America [phone]

http://www.operationaldynamics.com/