Re: [gentoo-server] backups/mirroring with rsync

[Jonathan Rogers]

Ben Munat wrote:
> Hello,
>
> So, I decided to keep things "simple" and just use rsync for backing up
> my web server. But now I'm wondering if "simple" was the right word...
> my head is spinning with all the possible configuration and command-line
> argument permutations. Below is my plan... if there are any rsync
> experts out there, I'd love to hear thoughts, warnings, advice, etc.

I'd recommend that you use rdiff-backup, since it is as simple to use as 
rsync and gives you the added benefit of file history, rather than only 
a snapshot. Configuring it is quite similar to rsync.

>
> I'm only using about 5 gigs on my server's drive. I have 50 gigs
> available on one partition alone on my home machine, so I'm planning on
> mirroring the server into a chroot on my home machine. My IP address has
> only changed once in the last four years, so that shouldn't be an issue.
>
> The initial sync will suck, but after that it shouldn't be too bad...
> not that much activity on this server. I'm figuring on setting up an
> hourly cron job on the server to run rsync. I'll also set up a user on
> my local machine just for this and have rsync chroot into that user's
> home directory.
>
> Then I'll just have the rsync start from "/" on the server. It doesn't
> seem like that much data to mess with picking and choosing. Plus, my
> hope is that if I should ever need this backup, I could just have the
> hosting company run my server from the gentoo cd and rsync the backup
> back onto the new drive.
>
> One question I have, however, is whether I need to exclude the /proc
> filesystem from the backup? Would that cause problems if I restored from
> the backup and then tried to boot the restored system? Or, hmmm,
> actually the /proc files are recreated on boot, right?

You should definitely exclude /proc. You probably also want to exclude 
/dev, especially if it is devfs or udev (which it almost certainly is). 
I also exclude /tmp, but that's up to you.

Another issue is data security. I exclude ".ssh" directories, since they 
contain private keys, which should never exist anywhere except where 
they were created. However, this is not an optimal setup, so I'm 
considering moving to a backup system that always stores everything 
encrypted.

>
> Someone told me that having the rsync server running on both machines
> makes things go faster... though I'm not clear on how you do that. Is
> this as simple as adding the "--server" option to the command? I was
> also planning on doing this over ssh to get encryption and to get
> through my firewall.

AFAIK, an rsync server has no inherent speed advantage. Setting up a 
server is how one makes files publicly available, such as the portage 
tree. However, you certainly don't want any of your data public.

You certainly want to use ssh or some other network transport that 
encrypts everything. This will impose processing overhead on both hosts, 
but it's absolutely essential for private data.

OpenSSH can use several block ciphers for stream encryption, several of 
which are much faster than the default. Google it for details. Here is 
some info on that:
<URL:http://www.linuxjournal.com/article.php?sid=6602>

>
> So, here's what I think I need to do:  Make my new user on my local
> machine (say, "backup-user", for example), giving him a home dir (/home
> is on it's own partition with 50 gigs free). Then I set up the
> rsyncd.conf file on my local machine with a module called, for example,
> "server-backup", give it the path "/home/backup", and say, "use
> chroot=yes". I suppose I could also put in a "hosts allow" clause with
> my server's IP address.

Again, I don't think there's any advantage to using rsyncd rather than 
just having rsync run from ssh; indeed, the two methods are mutually 
exclusive, AFAIK.

>
> Then, on the server, I put a script in the cron.hourly folder that just
> says something like:
>
> rsync -az -e ssh --delete --server --password-file=~secret /
> backup-user@<my ip address>::server-backup
>
> (but all on one line...)
>
> If I've got this right, that would tell rsync to use "archive" mode
> (preserving permissions and ownership) and compression. It would then
> connect over ssh to my home machine as "backup-user", getting the
> password from a (chmod 600!) file called "~secret", and sync everything
> under "/" on the server into "/home/backup-user" (which is the path for
> the "server-backup" rsync module), chrooted in that directory.
>

Rsync will only be able to preserve ownership and permissions if it runs 
as root on the backup machine. This is a security issue that is 
hopefully mitigated by running in a chroot, but consider it carefully.

> Oh crap... I just noticed that the man page says, if I'm understanding
> it correctly, that chroot isn't available unless I connect as root. That
> seems dumb... rsyncd is running as root on my machine, it should be able
> to chroot. Not that I really need the chroot I guess... I'm allowing
> only my server to connect, so I suppose I'll just skip that.
>
> Anyway, sorry this is so long... but I wanted to make sure I have this
> all correct before I go trying anything. Though, I suppose I'll use that
> "-n" dry-run option before actually doing this anyway.
>
> Well, thoughts and observations much appreciated...

I hope I can be of some help.

Jonathan Rogers