Gentoo Archives: gentoo-user

From: cal <cal@×××××××××.technology>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] is "scp" reliable?
Date: Sat, 29 May 2021 21:58:48
Message-Id: 8a5af6e4-0544-df31-98aa-58dbddfa1608@mail.meme.technology
In Reply to: Re: [gentoo-user] is "scp" reliable? by thelma@sys-concept.com
1 On 5/29/21 1:32 PM, thelma@×××××××××××.com wrote:
2 > On 5/29/21 2:00 PM, Mark Knecht wrote:
3 >>
4 >>
5 >> On Sat, May 29, 2021 at 12:54 PM <thelma@×××××××××××.com <mailto:thelma@×××××××××××.com>> wrote:
6 >>>
7 >>> I use "scp" to copy large file over local network, windows-7_pro.ova (about 28GB) but I it failed to import the appliance with VirtualBox.
8 >>> Checking the md5sum of the source and destination it turn out they are different.
9 >>>
10 >>> Is "scp" reliable for large files?
11 >>
12 >> In my experience scp is reliable but for large files I'd probably opt for some version of rsync, possibly the -P option. That way if the transfer stops for some reason you can generally pick up where it left off.
13 >>
14 >> HTH,
15 >> Mark
16 >
17 > Another mystery.
18 > I copied the file to USB 1TB sandisk.
19 > md5sum check OK same as my computer
20 >
21 >
22 > md5sum /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
23 > 6f3348f1fb915af9c45806d947558a37 /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
24 >
25 > I mount the same USB 1TB sandisk on another computer and running md5sum on same file gives me different number, why???
26 >
27 > md5sum /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
28 > c478cb48e2f7961cb0e3eb452df6e642 /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova
29 >
30 Did you sync and unmount the partition before ejecting the drive from
31 the first computer? With a file this large being copied, it is likely
32 that a large amount of data remains buffered/cached and will not be
33 fully written to the flash memory even after the copy command completes.
34
35 On the first machine, you would still see the correct md5sum because the
36 kernel abstracts this fact away from you. But if you rip out the drive
37 and take it somewhere else without flushing those caches, you're going
38 to get an incomplete file.
39
40 Check if the file on the drive still md5sums the same if you plug it
41 back into the first machine. Check what size it is, and whether there
42 are a lot of 0s at the end indicating an unfinished write.
43
44 cal

Replies

Subject Author
Re: [gentoo-user] is "scp" reliable? thelma@×××××××××××.com