1 |
On 3/8/21 6:03 am, Frank Steinmetzger wrote: |
2 |
> Am Mon, Aug 02, 2021 at 02:12:24PM +0800 schrieb William Kenworthy: |
3 |
>>>>> And you are storing several machines into a single repo? The docs say this |
4 |
>>>>> is not supported officially. But I have one repo each for /, /home and data |
5 |
>>>>> for both my PC and laptop. Using a wrapper script, I create snapshots that |
6 |
>>>>> are named $HOSTNAME_$DATE in each repo. |
7 |
>>>> Basicly yes: I use a once per hour snapshot of approximately 500Gib of |
8 |
>>>> data on moosefs, plus borgbackups 3 times a day to individual repos on |
9 |
>>>> moosefs for each host. |
10 |
>>> So you have: |
11 |
>>> Host A ──[hourly]──> Online-Borg A ─┐ |
12 |
>>> ├──[3/day]──> Offline-Borg |
13 |
>>> Host B ──[hourly]──> Online-Borg B ─┘ |
14 |
>>> ? |
15 |
>>> […] |
16 |
>> Hi Frank, |
17 |
>> |
18 |
>> Not quite - I see I could have been clearer. I "experiment" a lot - |
19 |
>> which means things break so I need to get back running quickly. So the |
20 |
>> purpose of the online repos and snapshots is just for that - quick |
21 |
>> recovery. |
22 |
> Whenever you say snapshot, you meen moosefs snapshots, right? Up until this |
23 |
> thread I’ve never heard of that FS. |
24 |
> |
25 |
> I would love to play more with storage systems, moving stuff around, backing |
26 |
> it up, assigning space and so on (basically play admin for a few people), |
27 |
> but apart from my ZFS-based NAS, I have nothing that would need this. I run |
28 |
> a nextcloud instance on my shared internet host and one on my raspi. That’s |
29 |
> as far as it gets. :D |
30 |
> |
31 |
>> stage 1: online, immediately available |
32 |
>> |
33 |
>> Hosts (those with actual attached storage - a mixture of intel, arm32 |
34 |
>> and arm64 devices are backed up to their own borg repo 3 times a day via |
35 |
>> push. One repo per machine on moosefs. |
36 |
>> |
37 |
>> A separate script does an hourly backup of VM, LXC images, and various |
38 |
>> data stores via a moosefs snapshot. |
39 |
>> |
40 |
>> stage 2: resource management for the snapshots |
41 |
>> |
42 |
>> 3 times a day, a script does a borg create on the latest snapshop at the |
43 |
>> time |
44 |
> So you mount the latest snapshot or access it in some other way and borg |
45 |
> *its* content, not the live data, right? |
46 |
Yes, though a moosefs snapshot is really a lazy copy of the data to a |
47 |
new location - issue the mfsmakesnapshot command and a few seconds later |
48 |
you have an identical copy of possibly terabytes of data in a new |
49 |
location with almost no extra disk space needed - though moosefs needs |
50 |
memory allocated to track the contents. i.e. think of it like |
51 |
symlink/hardlink to the original data until a file is changed whereupon |
52 |
its links are broken and its new data - its a little more complex than |
53 |
that but that's the gist of it. If you need data from a snapshot - you |
54 |
just copy it out or use it in place which breaks the link if written to. |
55 |
> |
56 |
>> and when complete deletes all previous snapshots (-1) so at that |
57 |
>> point I have two older snapshots available + a couple created during the |
58 |
>> borg run - note that large multi GiB snapshots can quickly use up all |
59 |
>> memory (32GiB) on the moosefs master unless culled regularly. |
60 |
> Sounds a bit delicate to me. If one link fails for some reason undetectedly, |
61 |
> you risk clog-up. |
62 |
Problems so far relate to borg failing to run for some reason - notice |
63 |
it and fix it, no problems overall as the rest keeps working |
64 |
> |
65 |
>> stage 3: offline because disasters happen :) |
66 |
>> |
67 |
>> All borg repos are on moosefs with a single root directory |
68 |
>> (/mnt/mfs/backups) so once every day or so I manually mount the offline |
69 |
>> disk and do a borg create on the backup directory. |
70 |
> What happens if that daily borg runs while the repos are being written to? |
71 |
|
72 |
To avoid this I mostly use fcron which has serialisation features so as |
73 |
long as I coordinate start and take into account run times its good. |
74 |
The manual copy is of course a bit tricky but again, its a timing |
75 |
issue. If I make a mistake, I would expect to include a repo that might |
76 |
need a check/repair before use. To borg, its just backing up files - it |
77 |
doesn't care that its another borg repo, in use or not. It still treats |
78 |
open files the same way as any other files - try and read them, but skip |
79 |
if unable to. Two borg instance cant work in the same repo, but one can |
80 |
and one can back that repo up at the same time because it just sees it |
81 |
as files if that makes sense. In reality, I have not needed to deal |
82 |
with it yet. The great thing about borg and the way this rota is |
83 |
structured is that I have a history I can go back to if necessary. In |
84 |
my experimenting in getting this right, I pay attention to the warnings |
85 |
borg issues and often delete the cache and security directories to make |
86 |
sure everything stays sane. |
87 |
|
88 |
Getting way off-topic here so we can take this off list if you are |
89 |
interested, or maybe others are interested here? |
90 |
|
91 |
BillK |
92 |
|
93 |
|
94 |
BillK |