Re: [gentoo-dev] Gentoo Enterprise deployment tools

[Brian Harring]

On Tue, Jun 21, 2005 at 08:35:52PM +0200, Thierry Carrez wrote:
> I don't say that it cannot be done, and I don't ask what's the best way
> to do it. I just ask *if* we should try to provide higher-level tools
> (and/or doc) to help in doing so. It's not obvious (especially for
> non-developers) how to proceed in that situation, even if a lot of
> people have designed their own solution in their corner.
Best way to do it?  Scary notion, not the way we're doing it 
currently.

Push mode is preferable imo, 'cept no code exists to support that.  
Someone could write the necessary client/server code, but that would 
have issues when bound into existing portage apis...

> >> With automatic deployments, would we run into difficult-to-solve
> >> etc-update problems ? Should/could the ServicePack system take care of
> >> that ?
> >
> >  I wouldn't use etc-update for this on a enterprise rollout personally.
> >    If you need config cfengine does a nice job, as well as using
> > cvs/rcs/something-else
>
> Again, the technology is out there, it's just not tightly integrated.
> Should we leave it as-is and let everyone design his own tools to
> connect the dots or should we ?
Not sure if the technology persay is out there honestly.  If it were a 
cluster, cloned boxes, I'd say minimalize CONFIG_PROTECT, and 
(assuming you write the client/server cruft above) slip in config pkgs 
that get installed alongside... or, just jam the config changes into 
the pkg (not clean but it's possible).  Or just trigger 
staggered reboot's on the boxes if you've got a fast network and pxe 
boot + imaging setup (I like the other method a bit more however :)

If you're managing a half dozen servers, each server running it's own 
customized httpd.conf, I don't see an easy way to handle that 
(would love to hear any ideas people have on that one).

Basically, kind of curious of how one could easily handle config 
management of multiple boxes, with config's potentially being wildly 
different from system to system (talking about a bit more then just 
/etc/conf.d/net.* and /etc/hostname differences here).  I suspect just 
wrapping the config changes into a bingpkg, and sliding them out 
alongside on a push would suffice, but that's just one possible 
method.

> >> Even in a simpler setup (preprod > production) we don't have the tools
> >> to push a software configuration change from a test machine to a
> >> production one.
> >
> > What exactly are you looking for here?
>
> Should we provide high-level software that defines an update pack (new
> binaries + configuration changes), allows to test it on a preproduction
> system and (once tested) to push it to registered production systems ?
> Or let everyone write his own treefreezing + network mounts + shell
> scripts + cfengine / CVS magic combo to do it ?
How do you push it?  I don't mean, what protocol/underlying, I'm 
asking how do you actually push _portage_ to do what you want?  Either 
you try and abuse the craptastic api in stable to pull it off, or you 
probably resort to a catalyst akin trick of calling emerge via system.

Neither is a proper solution.  Api is required, further, preferably 
portage innards being designed such that you can swap in your own 
remote subsystem (whether cache tree or config) so it's a matter of 
pushing commands down the client/server pipes, with the portage 
config/installation on that box pulling what it needs (remote tree == 
having to pull all relevant files if building, binpkg is easier 
however).

> >   Portage needs work; I know the devs are working on it, I know there
> > are other people who are doing there own things.  I see a lot of
> > portage-2.1 features that greatly simplify what you are trying to do (
> > repositories, config rewrite..etc.. ).  I think portage and what it
> > covers is a big part of this.  Recollecting a conversation with jstubbs
> > about portage he mentioned that he wouldn't want the portage-team to
> > maintain a Enterprise-like distribution program, but that the new API
> > would be great to write one against ;)
>
> I don't think it should be the role of the portage-team either.

I draw a slightly finer line... portaged, some hypothetical 
client/server ap, not our business to implement, just provide an api 
for them to use.  Thing is, if they're going remote, they'll either 
need to be able to trigger sync's on the boxes local tree 
(innefficient as all hell), or the tree is remote.  If the tree is 
remote, that falls on portage devs head to provide a framework so the 
tree can be remote, in other words abstraction/framework design.

Further... if you're pushing updates out, you need some method to 
query the vdb from the target- even if you're dealing with pushing 
updates down to a set of identical installations, you need to identify 
(easily/cleanly) what needs to be built, and what needs to be pushed 
down the line.  Dancing around it, but you need access to the vdb for 
that system definition, which probably would be stored locally... in 
which case, the system targets probably would need to have a remote 
vdb.

Implementing all of the crazy and fun stuff isn't portage (the 
project) business (interest in it personally, but other things have 
much higher priority).  To do the crazy/fun stuff requires a sane 
design so stuff can be swapped in/out as required, which falls on our 
heads though, and is what's being kicked around/worked on now.

> > I know Cardoe and genstef? are working on a seperate package manager
> > that just handles binaries but uses all the current portage stuff, so
> > you might want to talk to them as well.
>
> I sure hope they will comment on that thread :)
Kind of curious what they're attempting myself, since I've not heard 
much thus far...
~harring
-- 
gentoo-dev@g.o mailing list