Re: [gentoo-server] Best practices in managing large server groups

[Wendall Cada]

On Tue, 2007-05-22 at 11:54 -0500, Charles Duffy wrote:
> Thank you for your advice. It's good to hear that someone else is in a
> similar scenario.
>
> The only thing that's going to be a hard sell around here is Catalyst --
> our senior sysadmin (who is a considerably more seasoned Gentoo user
> than myself) has taken the position that using Catalyst is unnecessary
> and that emerge (targeting a chroot, with appropriate FEATURES set to
> generate binary packages) should be adequate. He's not immovable on such
> things -- but I'll need to have a very solid understanding of the
> benefits of using Catalyst before arguing any case to the contrary.
> (I've also contacted the author of gentoo-buildhoster to find out why he
> abandoned it, what he's using in its place, and whether he believes the
> project to be worth picking up; if he believes catalyst-2 to be more
> suitable for his use cases, understanding the reasoning behind that
> decision may provide some powerful arguments).
>
I very much agree with his assessment here. This is what I do for binary
distribution to desktop systems. It is very effective. I have played
with Catalyst quite a bit, but agree that catalyst-2 is much more well
suited for this.

I use also use the method described here: http://m8y.org/gentoosync.txt
to filter the portage tree. It makes for faster sync times, but
primarily, it makes me take a look at any additions or changes to the
tree. I've caught potential problems with updates by simply keeping a
very clean tree, which is important on the build server.

I surprises me how little work has gone into the binary support in
portage with the sheer number of devs around here who use the features.
I asked over three years ago about some specific bugs, and was assured
that they would be fixed soon. I don't think there has been a single
significant change in that time. Would be nice if -g/G was fixed. Here
is the method I use for using -k/K:

For package/*.tbz2 format:
#!/bin/bash
cd /usr/portage/packages/All/
all=`emerge -p $@ | grep "\[ebuild " | sed -e "s/\[ebuild[^]]*] //" | /
awk '{print $1}'`
for pkg in ${all}
do
    `wget -nc http://binhost.tld/portage/packages/${pkg}.tbz2`
done

For *.tbz2 format:
#!/bin/bash
cd /usr/portage/packages/All/
all=`emerge $@ -p | grep "\[ebuild " | sed -e "s/\[ebuild[^]]*] //" / |
sed -e "s/.*\///g" | awk '{print $1}'`
for pkg in ${all}
do
    `wget -nc http://binhost.tld/packages/All/${pkg}.tbz2`
done


Maybe someday, I'll turn it into a proper script, but for now, this
works.

Wendall

-- 
Only wimps use tape backup: _real_ men just upload their important stuff
on ftp, and let the rest of the world mirror it ;)
        -- Linus Torvalds