Gentoo Archives: gentoo-user

From: cal <cal@×××××××××.technology>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] is "scp" reliable?
Date: Sun, 30 May 2021 01:09:59
Message-Id: 8b72cb4a-b2ce-ae7f-bd5c-c4ff7038d30f@mail.meme.technology
In Reply to: Re: [gentoo-user] is "scp" reliable? by thelma@sys-concept.com
1 On 5/29/21 5:42 PM, thelma@×××××××××××.com wrote:
2 >>> Another mystery.
3 >>> I copied the file to USB 1TB sandisk.
4 >>> md5sum check OK same as my computer
5 >>>
6 >>>
7 >>> md5sum /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
8 >>> 6f3348f1fb915af9c45806d947558a37 /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
9 >>>
10 >>> I mount the same USB 1TB sandisk on another computer and running md5sum on same file gives me different number, why???
11 >>>
12 >>> md5sum /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
13 >>> c478cb48e2f7961cb0e3eb452df6e642 /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
14 >>>
15 >> Did you sync and unmount the partition before ejecting the drive from
16 >> the first computer? With a file this large being copied, it is likely
17 >> that a large amount of data remains buffered/cached and will not be
18 >> fully written to the flash memory even after the copy command completes.
19 >>
20 >> On the first machine, you would still see the correct md5sum because the
21 >> kernel abstracts this fact away from you. But if you rip out the drive
22 >> and take it somewhere else without flushing those caches, you're going
23 >> to get an incomplete file.
24 >>
25 >> Check if the file on the drive still md5sums the same if you plug it
26 >> back into the first machine. Check what size it is, and whether there
27 >> are a lot of 0s at the end indicating an unfinished write.
28 >>
29 >> cal
30 >
31 > Yes, I unmounted the USB device every time.
32 > And yes, I plug the USB device back to original machine and md5sum is correct, same as the original.
33 >
34 > I copied the large file over network to another box and md5sum of: windows-7_pro_May-23-21.ova is correct same as on the original box.
35 >
36 > I run this: "rsync -avh [source] [destination] && rsync -avhc [source] [destination]"
37 >
38 > above code rsync files folder on first run and if complete without issue, will run rsync again immediately while performing same file name comparison by using hash of entire file.
39 >
40 > This i what I got:
41 >
42 > rsync -avh windows-7_pro_May-29-21.ova fd@10.0.0.138:/home/fd/business/VDI/ && rsync -avhc windows-7_pro_May-29-21.ova fd@10.0.0.138:/home/fd/business/VDI/
43 > sending incremental file list
44 > windows-7_pro_May-29-21.ova
45 >
46 > sent 30.29G bytes received 35 bytes 115.81M bytes/sec
47 > total size is 30.28G speedup is 1.00
48 > sending incremental file list
49 > windows-7_pro_May-29-21.ova
50 > WARNING: windows-7_pro_May-29-21.ova failed verification -- update discarded (will try again).
51 > windows-7_pro_May-29-21.ova
52 > ERROR: windows-7_pro_May-29-21.ova failed verification -- update discarded.
53 >
54 > sent 33.44M bytes received 6.47M bytes 123.38K bytes/sec
55 > total size is 30.28G speedup is 758.62
56 > rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]
57 >
58 >
59 >
60 rsync is emitting errors indicating the file was not transferred
61 correctly. At this point I would call into question whether your second
62 machine is the problem rather than any of the tools you're using. If
63 you have a third machine that is easy to test. Otherwise I would run
64 memtest86+ and smartctl.
65
66 cal

Replies

Subject Author
Re: [SOLVED] [gentoo-user] is "scp" reliable? thelma@×××××××××××.com