[gentoo-user] Corruption of scp'd files

[Mick]

I am struggling to understand why this problem occurs:

I scp a number of binary files (e.g. videos) from box A to box B.  I have 
bz2'd them beforehand.  At box B I run bzip2 -tv file.bz2, which reports that 
all is OK.

Then I run bzip2 -dv file.bz2 but it tells me that some of the contents have 
been corrupted.  Fair enough, I run bzip2recover which runs its CRC checks 
and does not report anything being wrong.  So, I know run bzip2 -tv 
rec00*file.bz2 and it finds a number of files with "data integrity (CRC) 
error in data".

I run bzip2recover on box A and scp afresh copies of the affected rec00* 
files.  Some of the rec00* files seem to be copied without further 
corruption, but one or two report again corruption (on box B only).

Unable to understand why this is so, I decide to use vanilla ftp to another 
box (in a data center this time) to copy over the same files from box A.  
Then run ftp get from box B.  bzip2 -tv shows no more corruption (although 
this is not universal - some files still show corruption when tested on box 
B).

My ISP applies traffic shaping (especially on the obscure ssh port that I have 
chosen for the transmission) so some packets may be dropped, but this does 
not explain why ftp occasionally works.  Can you explain why this seemingly 
random corruption occurs?  Can I trust bzip2 and its CRC tests?
-- 
Regards,
Mick