Gentoo Archives: gentoo-user

From: Peter Humphrey <peter@××××××××××××.uk>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] How to compress lots of tarballs
Date: Tue, 28 Sep 2021 11:19:54
Message-Id: 5507298.DvuYhMxLoT@wstn
In Reply to: [gentoo-user] How to compress lots of tarballs by Peter Humphrey
1 On Sunday, 26 September 2021 11:57:43 BST Peter Humphrey wrote:
2 > Hello list,
3 >
4 > I have an external USB-3 drive with various system backups. There are 350
5 > .tar files (not .tar.gz etc.), amounting to 2.5TB. I was sure I wouldn't
6 > need to compress them, so I didn't, but now I think I'm going to have to.
7 > Is there a reasonably efficient way to do this? I have 500GB spare space on
8 > /dev/sda, and the machine runs constantly.
9
10 To complete the topic, and in case anyone's interested, I've settled on the
11 following method, which could have been made more general, but this way I
12 have a complete set of compressed tarballs in two places in case I need to
13 recover something.
14
15 This is the command to compress one week's backups:
16
17 # (cd /mnt/sdc/wstn/main/vvv.old && time (for tar_archive in *.tar; do zstd -
18 T0 --rm "${tar_archive}" -o /mnt/bu-space/vvv.old/"${tar_archive}.zst"; done
19 && cp /mnt/bu-space/vvv.old/* /mnt/sdc/wstn/main/vvv.old/ && sync))
20
21 This was 36GB and took 20 minutes.
22
23 --
24 Regards,
25 Peter.