Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-dev
Navigation:
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: mgorny@g.o
From: Brian Harring <ferringb@...>
Subject: Re: Re: metadata/md5-cache
Date: Mon, 4 Jun 2012 06:15:13 -0700
On Mon, Jun 04, 2012 at 09:27:10AM +0200, Micha?? G??rny wrote:
> On Sun, 3 Jun 2012 09:48:26 +0000
> "Robin H. Johnson" <robbat2@g.o> wrote:
> 
> > On Sun, Jun 03, 2012 at 11:34:07AM +0200, Micha?? G??rny wrote:
> > > I means using separate proto for metadata, not necesarrily git. In
> > > any case, if it comes to transferring a lot of frequently-changing
> > > files, rsync is not that efficient...
> > It does NOT send any of the intermediate states.
> 
> But it does have to check all the files.

Which is a pretty minimal cost in the grand scheme of things.  You 
also need to figure out what 'efficiency' you're going to talk about 
here; network io, disk io, cpu io, etc.  Most people in this case care 
about network IO; rsync's not perfect, but for reasons described 
below, it's the best of breed for the usage scenario.

> Did I mention I'm not talking necessarily about git?

Git would be sanest if you were after this; it already does point to 
point delta transformations sanely.  No point in reinventing a VCS; if 
you can't force the tree back to a known good state (aka, distributed 
VCS), you can't apply deltas to it, which case you need an rsync like 
algo.


> Rather anything which would just 
> lookup our timestamp, revision or whatever and just send what have 
> changed, in a packed manner.

This would be reinventing git/VCS, or more likely, pretending that a 
timestamp file automatically means the repository is *unmodified*, and 
trying to do a point to point transformation on it.  Where you're 
notion breaks down is that fun little bit about "unmodified".

This is why rsync is used; it's not limited to a point to point 
transformation, it's able to work from any starting point 
*efficiently*.

Either way, suggest you do some research into this- including 
efficiencies of rsync, git, existing snapshot delta rsync machinery 
(tarsync, diffball, etc), study the trade offs inherint in each.  Your 
initial email frankly reaks of NIH, hence my suggestions to go 
investigate what exists now.

~harring



References:
metadata/md5-cache
-- James Cloos
Re: metadata/md5-cache
-- Duncan
Re: Re: metadata/md5-cache
-- Robin H. Johnson
Re: Re: metadata/md5-cache
-- Robin H. Johnson
Re: Re: metadata/md5-cache
-- Michał Górny
Navigation:
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: Re: metadata/md5-cache
Next by thread:
Re: Re: metadata/md5-cache
Previous by date:
Re: [gentoo-commits] gentoo-x86 commit in x11-themes/gtk-engines-nodoka: ChangeLog
Next by date:
Re: Git braindump: 2 of N: developer interaction (merge co-ordinators)


Updated Jun 29, 2012

Summary: Archive of the gentoo-dev mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.