Re: [gentoo-user] Re: USB crucial file recovery

[Michael Mol]

On Wednesday, August 31, 2016 12:12:15 AM Volker Armin Hemmann wrote:
> Am 30.08.2016 um 23:59 schrieb Rich Freeman:
> > On Tue, Aug 30, 2016 at 4:58 PM, Volker Armin Hemmann
> >
> > <volkerarmin@××××××××××.com> wrote:
> >> the journal does not add any data integrity benefits at all. It just
> >> makes it more likely that the fs is in a sane state if there is a crash.
> >> Likely. Not a guarantee. Your data? No one cares.
> >
> > That depends on the mode of operation.  In journal=data I believe
> > everything gets written twice, which should make it fairly immune to
> > most forms of corruption.
>
> nope. Crash at the wrong time, data gone. FS hopefully sane.

No, seriously. Mount with data=ordered. Per ext4(5):

data={journal|ordered|writeback}
              Specifies the journaling mode for file data.  Metadata is always 
journaled.  To use modes other than ordered on the root filesystem, pass the 
mode to the kernel as boot parameter, e.g. rootflags=data=journal.

              journal
                     All data is committed into the journal prior to being 
written into the main filesystem.

              ordered
                     This is the default mode.  All data is forced directly 
out to the main file system prior to its metadata being committed to the 
journal.

              writeback
                     Data ordering is not preserved – data may be written into 
the main filesystem after its metadata has been committed to the journal.  This 
is rumoured to be the highest-throughput option.  It guarantees internal 
filesystem integrity, however it can allow old data to appear in files after a 
crash and journal recovery.



In writeback mode, only filesystem metadata goes through the journal. This 
guarantees that the filesystem's structure itself will remain intact in the 
event of a crash.

In data=journal mode, the contents of files pass through the journal as well, 
ensuring that, at least as far as the filesystem's responsibility is concerned, 
the data will be intact in the event of a crash.

Now, I can still think of ways you can lose data in data=journal mode:

* You mounted the filesystem with barrier=0 or with nobarrier; this can result 
in data writes going to disk out of order, if the I/O stack supports barriers. 
If you say "my file is ninety bytes" "here are ninety bytes of data, all 9s", 
"my file is now thirty bytes", "here are thirty bytes of data, all 3s", then in 
the end you should have a thirty-byte file filled with 3s. If you have barriers 
enabled and you crash halfway through the whole process, you should find a file 
of ninety bytes, all 9s. But if you have barriers disabled, the data may hit 
disk as though you'd said "my file is ninety bytes, here are ninety bytes of 
data, all 9s, here are thirty bytes of data, all 3s, now my file is thirty 
bytes." If that happens, and you crash partway through the commit to disk, you 
may see a ninety-byte file consisting of  thirty 3s and sixty 9s. Or things may 
landthat you see a thirty-byte file of 9s.

* Your application didn't flush its writes to disk when it should have.

* Your vm.dirty_bytes or vm.dirty_ratio are too high, you've been writing a 
lot to disk, and the kernel still has a lot of data buffered waiting to be 
written. (Well, that can always lead to data loss regardless of how high those 
settings are, which is why applications should flush their writes.)

* You've used hdparm to enable write buffers in your hard disks, and your hard 
disks lose power while their buffers have data waiting to be written.

* You're using a buggy disk device that does a poor job of handling power 
loss. Such as some SSDs which don't have large enough capacitors for their own 
write reordering. Or just about any flash drive.

* There's a bug in some code, somewhere.

>
> > f2fs would also have this benefit.  Data is not overwritten in-place
> > in a log-based filesystem; they're essentially journaled by their
> > design (actually, they're basically what you get if you ditch the
> > regular part of the filesystem and keep nothing but the journal).
> >
> >> If you want an fs that cares about your data: zfs.
> >
> > I won't argue that the COW filesystems have better data security
> > features.  It will be nice when they're stable in the main kernel.
>
> it is not so much about cow, but integrity checks all the way from the
> moment the cpu spends some cycles on it. Caught some silent file
> corruptions that way. Switched to ECC ram and never saw them again.

In-memory corruption of a data is a universal hazard. ECC should be the norm, 
not the exception, honestly.

-- 
:wq