Re: [gentoo-user] Re: [OT] fast recursive local copy

Nikos Chantziaras <realnc@×××××.de> wrote:

> On 08/14/2009 05:47 AM, meino.cramer@×××.de wrote:
> > Hi,
> >
> >   I am looking for a faster way to do a
> >
> >      cp -a r<thisdir>  <thatdir>
> >
> >   locally on one machine with one harddisk inside.
> >
> >   Is there a neat trick to accomplish this faster than
> >   good old cp?
>
> Nope.  Some people like to use pipes in hope to speed it up, something like:
>
>    tar -c <thisdir> | tar -xC <thatdir>
>
> but this isn't really faster and fscks up sparse files.

Pipes definitely do not speed up things, they slow things down as pipes 
introduce a significant system overhead.

The fastest method for copying directory trees (typicalls 30% faster than any 
other known method) is to use star:

star -copy -p -xdot -acl -sparse -C <fromdir> . <todir>

Do not forget the '.' !!!!!

Note that on Linux you may _need_ to add "-no-fsync" because file I/O is slow 
on Linux. On Solaris, not using -no-fsync slows things down by aprox. 10% but 
allows star to grant that everything was really copied to stable storage.
On Linux, ot using -no-fsync slows things down by aprox. 400%, this is why
I recommend to add "-no-fsync". BTW: other programs behave like star 
"-no-fsync" by default.

It may help to speed up things (in case you have enough RAM) to add: fs=128M
Use no more than 1/2 of the physical RAM as FIFO size.

Star implements the most effective way since more than 20 years. The idea is
to use a FIFO made of shared memory and to let star fork by default into two 
processes that work independently. 


Jörg

-- 
 EMail:joerg@××××××××××××××××××××××××.de (home) Jörg Schilling D-13353 Berlin
       js@××××××××××××.de                (uni)  
       joerg.schilling@××××××××××××××××.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily