Re: [gentoo-user] Re: USB crucial file recovery

[Rich Freeman]

On Wed, Aug 31, 2016 at 10:33 AM, Michael Mol <mikemol@×××××.com> wrote:
> On Wednesday, August 31, 2016 12:12:15 AM Volker Armin Hemmann wrote:
>> Am 30.08.2016 um 23:59 schrieb Rich Freeman:
>> >
>> > That depends on the mode of operation.  In journal=data I believe
>> > everything gets written twice, which should make it fairly immune to
>> > most forms of corruption.
>>
>> nope. Crash at the wrong time, data gone. FS hopefully sane.
>
> In data=journal mode, the contents of files pass through the journal as well,
> ensuring that, at least as far as the filesystem's responsibility is concerned,
> the data will be intact in the event of a crash.
>

Correct.  As with any other sane filesystem if you're using
data=journal mode with ext4 then your filesystem will always reflect
the state of data and metadata on a transaction boundary.

If you write something to disk and pull the power after fsck the disk
will either contain the contents of your files before you did the
write, or after the write was completed, and never anything
in-between.

This is barring silent corruption, which ext4 does not protect against.

> Now, I can still think of ways you can lose data in data=journal mode:

Agree, though all of those concerns apply to any filesystem.  If you
unplug an unmounted device, or pull the power when writes are pending,
or never hit save, or whatever, then your data won't end up on disk.
Now, a good filesystem should ensure that the data which is on disk is
completely consistent.  That is, you won't get half of a write, just
all or nothing.

>> >> If you want an fs that cares about your data: zfs.
>> >
>> > I won't argue that the COW filesystems have better data security
>> > features.  It will be nice when they're stable in the main kernel.
>>
>> it is not so much about cow, but integrity checks all the way from the
>> moment the cpu spends some cycles on it.

What COW does get you is the security of data=journal without the
additional cost of writing it twice.  Since data is not overwritten in
place you ensure that on an fsck the system can either roll the write
completely forwards or backwards.

With data=ordered on ext4 there is always the risk of a
half-overwritten file if you are overwriting in place.

But I agree that many of the zfs/btrfs data integrity features could
be implemented on a non-cow filesystem.  Maybe ext5 will have some of
them, though I'm not sure how much work is going into that vs just
fixing btrfs, or begging Oracle to re-license zfs.

>> Caught some silent file
>> corruptions that way. Switched to ECC ram and never saw them again.
>
> In-memory corruption of a data is a universal hazard. ECC should be the norm,
> not the exception, honestly.
>

Couldn't agree more here.  The hardware vendors aren't helping here
though, in their quest to try to make more money from those sensitive
to such things.  I believe Intel disables ECC on anything less than an
i7.  As I understand it most of the mainline AMD offerings support it
(basically anything over $80 or so), but it isn't clear to me what
motherboard support is required and the vendors almost never make a
mention of it on anything reasonably-priced.

If your RAM gets hosed than any filesystem is going to store bad data
or metadata for a multitude of reasons.  The typical x86+ arch wasn't
designed to handle hardware failures around anything associated with
cpu/ram.

The ZFS folks tend to make a really big deal out of ECC, but as far as
I'm aware it isn't any more important for ZFS than anything else.  I
think ZFS just tends to draw people really concerned with data
integrity, and once you've controlled everything that happens after
the data gets sent to the hard drive you tend to start thinking about
what happens to it beforehand.  I had to completely reinstall a
windows system not long ago due to memory failure and drive
corruption.  Wasn't that big a deal since I don't keep anything on a
windows box that isn't disposable, or backed up to something else.

-- 
Rich