Re: [gentoo-catalyst] catalyst enhancement

[Ramon van Alteren]

Chris Gianelloni wrote:
> On Mon, 2007-04-23 at 23:39 +0200, Ramon van Alteren wrote:
>
>> This puzzles me, if making a tool easier to use for it's users isn't a
>> valid reason for hacking at the tool, then what is ?
>>
> Well, I know I have said this before, but I'll say it again here.  We
> develop catalyst for our own usage first, and everyone else second.  If
> it doesn't directly impact Release Engineering, it immediately gets a
> back seat to changes that we need/want.  When I said "you" here, I meant
> you specifically, not any other form of you.
>
Fair enough and very clear.
I have to say that this is a much clearer rejection reason then "show me 
a use-case which can only be solved with relative paths in the specfile".
I'd be very hard-pressed to come up with such a use-case regardless of 
the tool.
>> It was a dual request, if nobody on list would be using the
>> functionality I'll maintain a patch outside the tree for our benefit.
>>
>
> This was really my question.  Would other people use it?
>
> One of the biggest problems that we have had with catalyst is people
> that want to change catalyst to meet their own specific needs and our
> need to balance things out so that we don't end up with unused code
> paths.  Having a single, consistent interface for the spec files allows
> for much simpler support on a product that we honestly wished we didn't
> have to support, at all.  If the change is something that lots of people
> would likely use, such as the stage4 target, then we will add it even if
> we don't use it ourselves.  Our general rule is don't change anything
> unless there is a really good reason.  As I said, simply making things
> slightly more convenient isn't really a good enough reason, IMO, unless
> a lot of people would use the functionality, and even then, it would
> depend on code availability and maintainability.  Of course, writing up
> a patch resolves the first issue, but the second would still remain.
>
Well let's see if there are others chiming in who want this 
functionality :-)

I'll give a short description what we do with catalyst just for the hell 
of it,
you might enjoy hearing what others do with the tool even though it's 
primarily developed for gentoo-releases.

Catalyst is one of the two gentoo projects which have a central role in 
our hands-off deployment system in our serverpark.
We prepare stage4 server images for our servers with catalyst for both 
amd64 and x86 archs.
We also generate development vmware images for our developers with it, 
which we try to keep as similar to production as we can get them.

We boot new servers with pxe and load them with a read-only nfs-root and 
a tmpfs overlay for host-specifics.
After that the install-tool by agaffney kicks in and installs the 
server-image created with catalyst on the new server.
We've modified install-tools to be able to look up install parameters in 
a mysql-database and use that to setup server-task specific parameters 
and networking.

After installing the server is rebooted and comes up with correct 
hostname and network settings.
Catalyst stage4 allows us to setup a few services to be started at boot, 
one of these is puppet a configuration client.
It connects to (one of) our puppet servers (aptly named puppetmasters 
:-) and loads the configuration for the services the server is supposed 
to perform.

The entire system helps us deploy new hardware:
Sticking a server in a rack in one of our datacenters to having it ready 
for production takes 30 minutes and we estimate (but haven't tested) 
that we can easily do 100 in an hour.

Catalyst is a key component in the system because it has made our build 
process for system images repeatable.
Before catalyst we were using chroots to build system images 
semi-manually, rebuilding a chroot to solve a bug after deployment was 
not a nice process to go through.
It's also removing a lot of bugs which were due to development vmware 
images being totally different from server-images.

We use the package-cache from catalyst to drive our binary package 
server and are investigating the use of the tinderbox target to generate 
repeatable builds for binary packages on top of the system-image 
versions we have deployed in our serverpark.

Right now we're busy converting our entire serverpark of 600 servers to 
catalyst-based server-images.
For a lot of server classes this is quick and painless re-install, for 
our data-intensive servers this is harder but we'll get there in the end.

If it wasn't clear from the above, we're extremely happy with catalyst :-)
If anyone is curious or wants to know more, feel free to poke me on irc, 
my nick is innocenti

Regards,

Ramon

-- 
gentoo-catalyst@g.o mailing list