| 1 |
On Thu, 30 Apr 2009 08:25:38 +0100 |
| 2 |
Neil Bothwick <neil@××××××××××.uk> wrote: |
| 3 |
|
| 4 |
> What backup medium are you using? |
| 5 |
|
| 6 |
Oh my, I've managed to forget about it! |
| 7 |
|
| 8 |
|
| 9 |
The medium is regular sata2 hard drives with ext3 filesystem on a |
| 10 |
dedicated backup machine with quite rusty debian (etch) linux. |
| 11 |
Most backed-up systems (that I care about) are actually freebsd 6, the |
| 12 |
rest are linux. Most stored backups are 20-60 GB. |
| 13 |
|
| 14 |
Main bottleneck here is the network - quite laggy 100 Mbps link, |
| 15 |
because this backup server is quite far and isolated from the rest. |
| 16 |
Also it's completely inaccessible from backed-up machines, aside from |
| 17 |
reverse tunnels, which I rarely use as a dirty hacks. |
| 18 |
And this link has tendency to go down every once in a while, |
| 19 |
interrupting ongoing transfers. |
| 20 |
|
| 21 |
That said, nightly backup should always be available, so the backups |
| 22 |
are actually created on hotswap sata2 drives of each individual |
| 23 |
machine and grabbed by backup server over ssh and, in some cases, nfs. |
| 24 |
|
| 25 |
These days the scripts on the backup server quite frequently (10-50 |
| 26 |
times a day) connect to the other hosts and receive requests for |
| 27 |
certain paths from stored backups. So they parse backups with tar |
| 28 |
picking out given paths and pushes them back, as requested. |
| 29 |
Needless to say, it is slow, hence the need. |
| 30 |
|
| 31 |
|
| 32 |
Well, that's probably a bit more verbose than necessary, but the point |
| 33 |
is that (I believe) the backups should be created right on backed-up |
| 34 |
systems' hard disks, so: |
| 35 |
1. Have random access to backup storage. |
| 36 |
2. Prolonged io/cpu load is a bad thing. |
| 37 |
3. Compression (at least of gzip ratio) is a must, because of limited |
| 38 |
storage on backed-up machines. |
| 39 |
4. In-backup seek times should be lower than tar (which is scanning |
| 40 |
the whole file). |
| 41 |
5. I write py scripts for a living, so the question is really in a |
| 42 |
backup format - transfer and storage structure is not the issue. |
| 43 |
|
| 44 |
Easiest thing I've thought of is just to generate "tar index" on first |
| 45 |
archive pass and then just skip to the recorded point in ungzipped |
| 46 |
stream, feeding the rest to tar, stopping when necessary, but there's |
| 47 |
no point to debug and maintain this system if there are better |
| 48 |
solutions already. |
| 49 |
|
| 50 |
|
| 51 |
> If hard disks, do you have a separate machine for storing them? If |
| 52 |
> so, BackupPC may suit your needs. The ebuild in Portage is out of |
| 53 |
> date but the latest version is available from Bugzilla - |
| 54 |
> http://bugs.gentoo.org/show_bug.cgi?id=141018 |
| 55 |
> |
| 56 |
> It allows restoration of individual files or directories, from any |
| 57 |
> backup point, and doesn't require any special software on the |
| 58 |
> machines being backed up, only SSH access. It can backup Windows and |
| 59 |
> *nix machines. |
| 60 |
|
| 61 |
Thanks, will check it out, but I'm afraid that live network backups |
| 62 |
aren't the best solution in my case. |
| 63 |
|
| 64 |
|
| 65 |
-- |
| 66 |
Mike Kazantsev // fraggod.net |
Archives