Gentoo Archives: gentoo-user

From: Walter Dnes <waltdnes@××××××××.org>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Use split to break up a 10GB file binary?
Date: Mon, 20 Jun 2011 21:16:36
Message-Id: 20110620205106.GB14933@waltdnes.org
In Reply to: [gentoo-user] Use split to break up a 10GB file binary? by Mark Knecht
1 On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote
2 > Hi,
3 > Is split an appropriate program to use to break a single 10GB file
4 > into 100 100MB files to transfer over the net using rsync, and then
5 > use cat to reassemble?
6 >
7 > Is there some better way to do this?
8
9 That's what split was written for. I can't think of anything better.
10 BTW, what type of data is the 10 gig file? If it's text, then consider
11 using zip or bzip2 on each of the fragments before transferring. If
12 it's an already compressed binary format, then don't waste time
13 attempting further compression.
14
15 --
16 Walter Dnes <waltdnes@××××××××.org>

Replies

Subject Author
Re: [gentoo-user] Use split to break up a 10GB file binary? Mark Knecht <markknecht@×××××.com>