Gentoo Archives: gentoo-user

From: John Covici <covici@××××××××××.com>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] sanoid (was Backup questions)
Date: Wed, 15 Aug 2018 10:54:56
Message-Id: m3zhxnrjbe.wl-covici@ccs.covici.com
In Reply to: Re: [gentoo-user] sanoid (was Backup questions) by "J. Roeleveld"
1 On Wed, 15 Aug 2018 02:45:20 -0400,
2 J. Roeleveld wrote:
3 >
4 > On Tuesday, August 14, 2018 10:52:30 PM CEST John Covici wrote:
5 > > On Tue, 14 Aug 2018 16:06:21 -0400,
6 > >
7 > > J. Roeleveld wrote:
8 > > > On August 14, 2018 11:42:18 AM UTC, John Covici <covici@××××××××××.com>
9 > wrote:
10 > > > >I use sanoid/syncoid to back up using zfs. Its great, keeps snapshots
11 > > > >for as long as I want them (I use 80 days for now). And it keeps
12 > > > >hourlies for the last couple of days as well, so I could roll back in
13 > > > >case of a problem. Very nice if you use zfs.
14 > > >
15 > > > I tried sanoid, but it has a few problems which really become annoying
16 > > > when you have a lot of datasets: 1) every dataset is handled seperately,
17 > > > no use of recursive snapshots when datasets are inside the same tree 2)
18 > > > it keeps seperate hourly, daily,.... snapshots, which means it will
19 > > > happily create multiple snapshots with only a few seconds difference for
20 > > > every dataset around midnight. 3) when rolling back several snapshots,
21 > > > there are multiple errors reported because the cache (where does it store
22 > > > that?) does not match reality.
23 > > >
24 > > > Have these been resolved yet?
25 > > >
26 > > > I ended up writing my own system for this, got some extra intelligence in
27 > > > there to work around any possible error condition I have encountered.
28 > > Well, I got around your second point by having a special job at 11:59
29 > > pm to create the dailies and the one at midnight works well. I only
30 > > do the cron jobs hourly, not every minute like they wanted.
31 > >
32 > > If your script is not special for you, I would like to see it, maybe I
33 > > would use it instead. Things seem to work for now, however with those
34 > > modifications.
35 >
36 > Current code is too specific for my situation. I think you would be quicker to
37 > write something yourself instead of modifying my code.
38 >
39 > The steps are:
40 > - Check if an snapshot needs to be done (incl. type: hourly, daily, weekly,
41 > monthly)
42 > - If yes:
43 > 1) create a recursive snapshot on the entire pool
44 > 2) remove unnecessary snapshots (temp, swap,
45 > placeholders,not_for_current_type)
46 > 3) register snapshots into database
47 > 4) clean up old snapshots
48 >
49 > Because I register actual snapshots and point snapshot-type entries to these,
50 > I can quickly determine which snapshots are really unecessary. This also
51 > drastically reduces the amount of snapshots on the system. (My SAN currently
52 > has 2611 ZFS snapshots).
53 >
54 > I also found it is far quicker to create a recursive snapshot on the entire
55 > pool and then removing all the unecessarily created ones.
56
57 OK, I will look into that.
58
59 --
60 Your life is like a penny. You're going to lose it. The question is:
61 How do
62 you spend it?
63
64 John Covici wb2una
65 covici@××××××××××.com