Re: [gentoo-server] Re: Gentoo build server

[Sancho2k.net Lists]

Joby Walker wrote:

> It is a defunction in portage and the methodology of the package.  There
> are a couple bug report monitoring this.  The portage developers promise
> that eventually base layout will be package safe.
>
>> ---How to handle the baselayout defunction--
>
>
> Actually it is easier than you suspect.  As long as you don't create a
> baselayout package, when you "emerge -k world" or "emerge -g world"
> portage will use all of the binary packages you have created and will
> build baselayout normally.  If you run "emerge -pk ...", portage will
> label tell you whether each piece of software is going to be installed
> via binary or ebuild.

This is basically the way we are working now.

What we've been able to do so far, (thanks for the input on this part) 
is to set up a build server where packages are built and distributed 
from. We now share out the tbz2s with apache2 and have PORTAGE_BINHOST 
set on "client" machines.

This setup seems to be working pretty good on existing installations. 
For new installs, it's a little less smooth than I had imagined.

If experience serves, the ability to use -g reliably is absent in the 
version of sys-apps/portage that comes on our boot media. That means 
that after extracting the system from a stage tarball, we have to emerge 
sync and then emerge -u portage to get us to the point that we can pull 
any more packages over with emerge -gk. Once sys-apps/portage is up to 
date, we can emerge -gk several more packages (about 1/3 to 1/2 of what 
emerge system -vp shows.) When we get somewhere around bash, wget stops 
working because it is linked against openssl-0.9.7 and only 
openssl-0.9.6 libraries are installed on the system (strange...). So we 
have to manually copy openssl package into packages/All, update openssl, 
and then continue. We can get pretty much through the rest of an emerge 
system after this.

The reason I relate this is that I would like to get input on whether 
our process can be streamlined past this point. Our goals are as follows:

  1. Centralize build source
  2. Rapid deployment (installation and packages)
  3. Control software/system versions

The intention is to set up a local rsync mirror on the LAN and use this 
as the source of updating portage across all systems. Then even if 
someone gets a wild hair and decides to emerge sync out of the blue, 
they have the latest portage available per our discretion. Then our 
build server and all "clients" will update thier Portage trees from this 
mirror. All packages will be compiled and built on the build server and 
possibly burned in prior to rollout on the production servers. This 
seems to fit the need of #1 and #3. #2 is proving to be a bit more 
difficult, as described above. Maybe if we used a more recent livecd 
(currently 1.4.? from 09/03) it would have a more current portage and 
allow us to take the system from untarring stage3 directly to emerge 
sync and system in less time.

>>> 2) Not running 'emerge sync' from each server is actually harmful to the
>>> server.  You might be able to use 'emerge regen' to mitigate most of
>>> this, but I haven't investigated it that fully (since I switched from
>>> NFS /usr/portage to a BINHOST).

Taking this advice to heart. I now see that trying to share /usr/portage 
across all hosts via NFS was foolish as Portage was not designed to 
operate like that.

> An "emerge sync" runs a lot of maintenance on the /var/db/pkg and
> /var/cache/edb directories.  As packages are relabeled or recategorized,
> "emerge sync" re-orgs your /var caches to reflect the new structure.  If
> this maintenance doesn't happen you will start getting dependancy errors
> when trying to install a package.  This happened to me with Perl.

I've heard recommendations to run fixpackages at times. What is the 
exact purpose of that? It appears to handle moving packages to different 
categories as Portage changes. Is it recommended to run this often, or 
after a new emerge sync, for example?

Another thing we're investigating is the need to use emerge with the -D 
and -U options to make sure that all packages and dependencies on the 
build server are always kept current. At this point, it sounds like a 
fine idea to me. Are there compelling reasons why this might not be a 
good idea?

Suggestions on how to accomplish our goals and streamline this process 
are happily taken.

Thanks for the input thusfar, it's very appreciated.

DS