[gentoo-server] SSH, updates and serial port switches

[Sébastien Arnaud]

Hi,

I have started to "pour" more Gentoo Linux based server in a datacenter 
over the past year, I lost control 3 times of remote servers. One of 
them was after a hard reboot and filesystem check which required to 
press a key on the physical machine, and the two remaining ones were 
linked to SSH terminating the connection after running some updates.
So, I wanted to get some advice on how you all handle keeping control 
of your remote Gentoo servers, and for instance how to keep SSH running 
at all costs.

I have seen in different FAQs that running a serial cable to each 
server and using a SSH serial console switch is a good idea, but I am 
having trouble finding something cheap in this arena. Also, how much 
better is it in terms of reliability in case something goes really 
wrong with the server? FYI, all the servers are plugged into a remote 
APC reboot switch but I almost never use this, as many times it ends up 
invalidating the filesystem and therefore requiring a physical 
intervention at the keyboard. Anyway around this problem as well?

Thank you in advance.

Sébastien

"The wise man doesn't give the right answers, he poses the right 
questions."
-- Claude Levi-Strauss