Re: [gentoo-user] How to compress lots of tarballs

[Rich Freeman]

On Wed, Sep 29, 2021 at 5:48 PM Wols Lists <antlists@××××××××××××.uk> wrote:
>
> An LVM snapshot creates a "copy on write" image. I'm just beginning to
> dig into it myself, but I agree it's a bit confusing.

So, snapshots in general are a solution for making backups atomic.
That is, they allow a backup to look as if the entire backup was taken
in an instant.

The simplest way to accomplish that is via offline backups.  Unmount
the drive, mount it read-only, then perform a backup.  That guarantees
that nothing changes between the time the backup starts/stops.  Of
course, it also can mean that for many hours you can't really use the
drive.

Snapshots let you cheat.  They create two views of the drive - one
that can be used normally, and one which is a moment-in-time snapshot
of what the drive looked like.  You backup the snapshot, and you can
use the regular drive.

With something like LVM you probably want to unmount the filesystem
before snapshotting it.  Otherwise it is a bit like just pulling the
plug on the PC before doing an offline backup - the snapshot will be
in a dirty state with things in various state of modification.  This
is because LVM knows nothing about your filesystem - when you run the
snapshot it just captures the state of every block on the disk in that
instant, which means dirty caches/etc (the filesystem caches or file
buffers wouldn't be covered, because those are above the LVM layer,
but caches below LVM would be covered, like the physical disk cache).

Some filesystems like btrfs and zfs also support snapshotting.  Those
are much safer to snapshot while online, since the filesystem is aware
of everything going on.  Of course even this will miss file buffers.
The cleanest way to snapshot anything is to ensure the filesystem is
quiescent.  This need only be for a few seconds while snapshotting
though, and then you can go right back to using it.

If you do a full backup on a snapshot, then if you restore that backup
you'll get the contents of the filesystem at the moment the snapshot
was taken.

You can also use snapshots as a sort-of backup as well.  Of course
those are on the same physical disk so it only protects you against
some types of failures.  Often there is a way to serialize a snapshot,
perhaps incrementally - I know zfs and btrfs both allow this.  This
can be a super-efficient way to create backup files since the
filesystem metadata can be used to determine exactly what blocks to
back up with perfect reliability.  The downside of this is that you
can only restore to the same filesystem - it isn't a simple file-based
solution like with tar/etc.

So as to not do a second reply I want to address an earlier question,
regarding backups that span multiple volumes.  This is a need I have,
and I haven't found a simple tool that does it well.  I am VERY
interested in suggestions here.  Tar I think can support something
like this, since it is common for tape backups to span multiple
volumes.  I have no idea how clean that is.  Most tools want you to
use LVM or a union filesystem or whatever to combine all the drives
into a single mountpoint and let linux manage the spanning.  I'm using
bacula which can sort-of handle multiple volumes, but not very
cleanly, and it is an overly complex tool for backing up one system.
I wouldn't really endorse it wholeheartedly, but it can be made to
work.

-- 
Rich