From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 2AE8F158042 for ; Sat, 9 Nov 2024 14:43:55 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 20792E088A; Sat, 9 Nov 2024 14:43:49 +0000 (UTC) Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id D6818E0805 for ; Sat, 9 Nov 2024 14:43:48 +0000 (UTC) Received: by mail-il1-f181.google.com with SMTP id e9e14a558f8ab-3a4e5a7b026so10721625ab.3 for ; Sat, 09 Nov 2024 06:43:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731163428; x=1731768228; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QKNSxoW82Se54U1GtPxNCyUFivb7dPRTiBFkYLLjzBw=; b=cCNbm4XZEzqrwddQ59uH8pSXGDi3eWGtExe3YQWAVM1vdltM+rCN2eXFyUgySNHe58 uhBmQB16TG2VE5DelORZ0izF7cVjtdpLZ1pgbf+r4fKSCDkOXp5tlsqxredw/uDOtK7F HGnpsaHCvgZe/cakV6ZKZNg9nZldqSPX/BDd0doiKC+hVihyPuw9e8vwKMWHFuzgwqBs 2msqeOD/fT/xwlxLLyeabrPiMgcYxw+816y2utx+sFuqQjQtkxjI3V8hvnaMXzjcuxbM qxKaes5er1IGewGm3XQgq7vJqfgz5RTzE9kxSXhx3rxHKv78G7/KyfDplyBjv4LA4yMk RuCg== X-Gm-Message-State: AOJu0YzbcEHmqNotUe+fecjYBPB1luB9llIViXNHDCnkhoN+eldnLcNx NksdF1gR7u2UmYpIaY84UpVerVuAct8np+7jA046RCbeTGBPxIjeQ9dWjL6UbaKo03RX5XpjLRB dMz+bj7D5NVceMjTygjdn7QJ+xmk0ZQ== X-Google-Smtp-Source: AGHT+IH0Se4BIX3Hj9rIYYkZ0aSHo3hClWgaYWwGeFwbvwvQhxpFKb9t5TXHXx7MupAUaaCluENyE+rZYT3BUikuwIM= X-Received: by 2002:a05:6e02:1aa3:b0:3a6:c122:508e with SMTP id e9e14a558f8ab-3a6f1a48b4emr70645055ab.17.1731163427728; Sat, 09 Nov 2024 06:43:47 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <713323a1-bd58-4607-898f-db2aedc9942c.ref@yahoo.com> <713323a1-bd58-4607-898f-db2aedc9942c@yahoo.com> In-Reply-To: <713323a1-bd58-4607-898f-db2aedc9942c@yahoo.com> From: Rich Freeman Date: Sat, 9 Nov 2024 09:43:43 -0500 Message-ID: Subject: Re: [gentoo-user] backup horror story (happy ending) To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 4cbbd7ca-99d8-4ca9-bbf0-6757d2659311 X-Archives-Hash: 2c6ac461188429de6e06e5a9deb7fa0e On Sat, Nov 9, 2024 at 5:30=E2=80=AFAM ralfconn wro= te: > > But I have a backup, no problem... till I realize the cron job had > already run so it had overwritten the old files with the new, corrupt > versions. > I highly recommend having multiple backups to avoid this sort of problem. If you aren't wedded to rsync, then restic seems to be the platform of choice these days. Not sure offhand if it handles ntfs very well. I use duplicati for backing up windows hosts to an S3 backend and that works great, but that is more of a windows solution (VSS and so on). I imagine that restic doesn't care much about the filesystem if it is running on linux and everything is mounted. If you are wedded to the rsync approach where your backups are just a big directory tree that you can easily access, then I suggest using rsnapshot. It is basically a wrapper around rsync that maintains a backup history, in a very clean way. Basically it does a hard-link copy of your entire backup set to a new directory tree named by the timestamp, and then it runs rsync to sync that new tree. The result is that you get file-level deduplication effectively, and otherwise get what looks like a nice big full copy of the backup source in each directory. It probably won't be as efficient as something like restic since I'm guessing that can do deduplication below the file level (I think it can also deduplicate across multiple hosts/etc if you're using it that way). Most of these modern tools still use librsync under the hood so the actual data transfer is just as efficient. rsync by itself is nice for its simplicity, but it just isn't a very elegant backup solution. You can tell it to preserve old file versions, but those end up stored next to the original files with different filenames and that can be a real mess to restore if you don't want to end up with all those old versions. With restic/rsnapshot you can go back in time but still get a clean restore, and you can still extract individual files from various snapshots. If you really are running rsync at any kind of scale also consider rclone, which is often faster since it can transfer multiple files in parallel, which is useful if you're more bound by latency than disk IOPS (often the case on solid state drives over networks). --=20 Rich