1 |
On Monday 20 Jun 2011 22:06:59 Mark Knecht wrote: |
2 |
> On Mon, Jun 20, 2011 at 1:51 PM, Walter Dnes <waltdnes@××××××××.org> wrote: |
3 |
> > On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote |
4 |
> > |
5 |
> >> Hi, |
6 |
> >> Is split an appropriate program to use to break a single 10GB file |
7 |
> >> into 100 100MB files to transfer over the net using rsync, and then |
8 |
> >> use cat to reassemble? |
9 |
> >> |
10 |
> >> Is there some better way to do this? |
11 |
> > |
12 |
> > That's what split was written for. I can't think of anything better. |
13 |
> > BTW, what type of data is the 10 gig file? If it's text, then consider |
14 |
> > using zip or bzip2 on each of the fragments before transferring. If |
15 |
> > it's an already compressed binary format, then don't waste time |
16 |
> > attempting further compression. |
17 |
> > |
18 |
> > -- |
19 |
> > Walter Dnes <waltdnes@××××××××.org> |
20 |
> |
21 |
> Hi Walter. |
22 |
> |
23 |
> It's a Virtualbox VM exported from virtualbox-4.0.8. It consumes about |
24 |
> 40GB on disk, compressed by Virtualbox to about 10GB by their 'Export |
25 |
> Appliance' feature. |
26 |
> |
27 |
> I actually did try compressing the exported 10GB file with gzip & |
28 |
> bzip2 before splitting it. None of those provided any compression. I |
29 |
> didn't try the spilt outputs as I figured they are just binary chunks |
30 |
> and wouldn't compress either. |
31 |
> |
32 |
> Thanks for the info. |
33 |
|
34 |
You could try: |
35 |
|
36 |
wget -c -t 0 |
37 |
|
38 |
to retry (forever) should the transfer fail for some reason and not bother |
39 |
with splitting the file. |
40 |
|
41 |
-- |
42 |
Regards, |
43 |
Mick |