Re: [gentoo-user] kernel notification of file system changes

[Iain Buchanan]

On Wed, 2010-05-05 at 18:35 +0100, Stroller wrote:
> On 5 May 2010, at 18:24, Florian Philipp wrote:
> >> ...
> >> man inotify(7):
> >> ... When a directory is monitored, inotify will return events for the
> >> directory itself, and for files inside the directory.
> >> ...
> >
> > To repeat my comment on Iain's original "backup to a cold-swap drive"
> > thread ...
>
> Sorry, I started ignoring that almost immediately it was posted.

ooohh, ouch! :)

>  He
> rejected too quickly too many workable solutions to basically
> functional backup. Perhaps Iain is a perfectionist, but I did not wish
> to follow the thread.

Perhaps I am a bit of a perfectionist, but I think you misunderstood my
aim.  I rejected 3 options straight away (dd, gparted, and Ghost4Linux)
because they're not designed for backing up a live filesystem in a
"change only" fashion ("intelligent" is the word I used).

Beyond that I didn't reject anything else that anyone mentioned.

> > ... Inotify has two drawbacks which make it hard or even impossible
> > to use for Iain's use case:
> >
> > a) It does not work recursively which means that you have to create a
> > new handle for each subdirectory. Of course, this only means more work
> > for the programmer but there is also the problem that
>
> Pardon me. I assumed that "files inside the directory" meant that foo
> would be be changed when foo/bar changed, thus monitoring grunt would
> reflect changes in grunt/foo/bar. I overlooked that a directory is not
> a file.

isn't it?  I thought a directory was just a file containing names (or
inodes) of other files?  Which would explain why monitoring grunt
wouldn't show changes in grunt/foo/bar, since the directory/file called
grunt remains the same (ie. contains the same list of inodes) even if
grunt/foo/bar changes.

Let me tell you what I actually want to do, which I may not have made
clear originally:

I want to backup root to an external drive (or that could be rephrased
as "I want to backup any mount to any other mount), such that:
     1. My backup is an hour or so out of date (at most)
     2. I don't need to copy the entire filesystem every time

To do that, I could either:
      * Run rsync every hour over the entire filesystem (I'm doing this
        now with ionice, takes about 10 minutes when there are no
        changes)
      * Use some file notification monitor to tell me which file was
        just changed, and only rsync that file

The problems with rsync is that during the rsync process, the filesystem
is changing, so I will end up with a slightly inconsistent backup.  If I
use some notification method that tells me a file has changed, I can
greatly reduce any inconsistency, and I reduce my hour down to seconds
or minutes, depending on how much changes at any one time.

I'm considering LVM for it's snapshot capability, but I'd still have to
rsync root.  I would prefer a file notification method as well, so I can
just rsync the file that just changed.

So far all the file monitoring tools are based on individual files (even
the recursive ones), and you eventually reach a system limit.

Thanks, and willing to listen to any ideas from anyone (except
Stroller :p )

-- 
Iain Buchanan <iaindb at netspace dot net dot au>

Usually, when a lot of men get together, it's called a war.
		-- Mel Brooks, "The Listener"