1 |
On Mon, Jun 20, 2011 at 1:51 PM, Walter Dnes <waltdnes@××××××××.org> wrote: |
2 |
> On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote |
3 |
>> Hi, |
4 |
>> Is split an appropriate program to use to break a single 10GB file |
5 |
>> into 100 100MB files to transfer over the net using rsync, and then |
6 |
>> use cat to reassemble? |
7 |
>> |
8 |
>> Is there some better way to do this? |
9 |
> |
10 |
> That's what split was written for. I can't think of anything better. |
11 |
> BTW, what type of data is the 10 gig file? If it's text, then consider |
12 |
> using zip or bzip2 on each of the fragments before transferring. If |
13 |
> it's an already compressed binary format, then don't waste time |
14 |
> attempting further compression. |
15 |
> |
16 |
> -- |
17 |
> Walter Dnes <waltdnes@××××××××.org> |
18 |
|
19 |
Hi Walter. |
20 |
|
21 |
It's a Virtualbox VM exported from virtualbox-4.0.8. It consumes about |
22 |
40GB on disk, compressed by Virtualbox to about 10GB by their 'Export |
23 |
Appliance' feature. |
24 |
|
25 |
I actually did try compressing the exported 10GB file with gzip & |
26 |
bzip2 before splitting it. None of those provided any compression. I |
27 |
didn't try the spilt outputs as I figured they are just binary chunks |
28 |
and wouldn't compress either. |
29 |
|
30 |
Thanks for the info. |
31 |
|
32 |
Cheers, |
33 |
Mark |