Gentoo Archives: gentoo-user

From: Frank Steinmetzger <Warp_7@×××.de>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] [OT] SMR drives (WAS: cryptsetup close and device in use when it is not)
Date: Mon, 02 Aug 2021 22:04:08
Message-Id: YQhrxEYvGs2TqUF3@kern
In Reply to: Re: [gentoo-user] [OT] SMR drives (WAS: cryptsetup close and device in use when it is not) by William Kenworthy
1 Am Mon, Aug 02, 2021 at 02:12:24PM +0800 schrieb William Kenworthy:
2 > >>> And you are storing several machines into a single repo? The docs say this
3 > >>> is not supported officially. But I have one repo each for /, /home and data
4 > >>> for both my PC and laptop. Using a wrapper script, I create snapshots that
5 > >>> are named $HOSTNAME_$DATE in each repo.
6 > >> Basicly yes: I use a once per hour snapshot of approximately 500Gib of
7 > >> data on moosefs, plus borgbackups 3 times a day to individual repos on
8 > >> moosefs for each host.
9 > > So you have:
10 > > Host A ──[hourly]──> Online-Borg A ─┐
11 > > ├──[3/day]──> Offline-Borg
12 > > Host B ──[hourly]──> Online-Borg B ─┘
13 > > ?
14 > > […]
15 >
16 > Hi Frank,
17 >
18 > Not quite - I see I could have been clearer.  I "experiment" a lot -
19 > which means things break so I need to get back running quickly.  So the
20 > purpose of the online repos and snapshots is just for that - quick
21 > recovery.
22
23 Whenever you say snapshot, you meen moosefs snapshots, right? Up until this
24 thread I’ve never heard of that FS.
25
26 I would love to play more with storage systems, moving stuff around, backing
27 it up, assigning space and so on (basically play admin for a few people),
28 but apart from my ZFS-based NAS, I have nothing that would need this. I run
29 a nextcloud instance on my shared internet host and one on my raspi. That’s
30 as far as it gets. :D
31
32 > stage 1: online, immediately available
33 >
34 > Hosts (those with actual attached storage - a mixture of intel, arm32
35 > and arm64 devices are backed up to their own borg repo 3 times a day via
36 > push.  One repo per machine on moosefs.
37 >
38 > A separate script does an hourly backup of VM, LXC images, and various
39 > data stores via a moosefs snapshot.
40 >
41 > stage 2: resource management for the snapshots
42 >
43 > 3 times a day, a script does a borg create on the latest snapshop at the
44 > time
45
46 So you mount the latest snapshot or access it in some other way and borg
47 *its* content, not the live data, right?
48
49 > and when complete deletes all previous snapshots (-1) so at that
50 > point I have two older snapshots available + a couple created during the
51 > borg run - note that large multi GiB snapshots can quickly use up all
52 > memory (32GiB) on the moosefs master unless culled regularly.
53
54 Sounds a bit delicate to me. If one link fails for some reason undetectedly,
55 you risk clog-up.
56
57 > stage 3: offline because disasters happen :)
58 >
59 > All borg repos are on moosefs with a single root directory
60 > (/mnt/mfs/backups) so once every day or so I manually mount the offline
61 > disk and do a borg create on the backup directory.
62
63 What happens if that daily borg runs while the repos are being written to?
64
65 --
66 Grüße | Greetings | Qapla’
67 Please do not share anything from, with or about me on any social network.
68
69 Even a Bonsai dreams of greatness.

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies

Subject Author
Re: [gentoo-user] [OT] SMR drives (WAS: cryptsetup close and device in use when it is not) William Kenworthy <billk@×××××××××.au>