Re: [gentoo-amd64] Re: btrfs Was: Soliciting new RAID ideas

[Marc Joliet]

Am Thu, 29 May 2014 06:41:14 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@×××.net>:

> Marc Joliet posted on Wed, 28 May 2014 22:32:47 +0200 as excerpted:
>
> > (Dammit, it seems that I've developed a habit of writing somewhat
> > long-winded emails :-/ . Sorry!)
>
> You?  <looking this way and that>  What does that make mine? =:^)

Novels, duh ;-) .

> > Am Wed, 28 May 2014 08:29:07 +0100 schrieb thegeezer
> > <thegeezer@×××××××××.net>:
> >
> >> top man, thanks for detail and the tips !
> >
> > I second this :) . In fact, I think I'll link to it in my btrfs thread
> > on gentoo-user.
>
> Thanks.  I was on the user list for a short time back in 2004 when I
> first started with gentoo, but back then it was mostly x86, while my
> interest was amd64, and the amd64 list was active enough back then that I
> didn't really feel the need for the mostly x86 user list, so I
> unsubscribed and never got around to subscribing again, when the amd64
> list traffic mostly dried up.  But if it'll help people there... go right
> ahead and link or repost.

I ended up simply forwarding it, as opposed to bumping my inactive thread.

> (Also, anyone who wants to put it up on the
> gentoo wiki, go ahead.  I work best on newsgroups and mailing lists, and
> find wikis, like most of the web, in practice read-only for my usage.
> I'll read up on them, but somehow never get around to actually writing
> anything on them, even if it would in theory save me a bunch of time
> since I could write stuff once and link it instead of repeating on the
> lists.)

Heh, the only Wiki I ever edited was at my old student job.  But yeah, I don't
feel comfortable enough in my BTRFS knowledge to write a Wiki entry myself.

> > I do have a question for Duncan (or anybody else who knows, but I know
> > that Duncan is fairly active on the BTRFS ML), though:
> >
> > How does btrfs handle checksum errors on a single drive (or when
> > self-healing fails)?
> >
> > That is, does it return a hard error, rendering the file unreadable, or
> > is it possible to read from a corrupted file?
>
> As you suspect, it's a hard error.

Damn >:-( .

> There has been developer discussion on the btrfs list of some sort of
> mount option or the like that would allow retrieval even with bad
> checksums, presumably with dmesg then being the only indication something
> was wrong, in case it's a simple single bit-flip or the like in something
> like text where it should be obvious, or media, where it'll likely not
> even be noticed, but I've not seen an actual patch for it.  Presumably
> it'll eventually happen, but to now there's a lot more potential features
> and bug fixes to code up than developers and time in their days to code
> them, so no idea when.  I guess when the right person gets that itch to
> scratch.

That's really too bad, I guess this isn't a situation that often arises for
BTRFS users.

> Which is yet another reason I have chosen the raid1 mode for both data
> and metadata and am eagerly awaiting the N-way-mirroring code in ordered
> to let me do 3-way as well, because I'd really /hate/ to think it's just
> a bitflip, yet not have any way at all to get to it.
>
> Which of course makes it that much more critical to keep your backups as
> current as you're willing to risk losing, *AND* test that they're
> actually recoverable, as well.

Of course, but like I said, I can't back up this one data partition.  I do have
backups for everything on my desktop computer, though, which are on the other
partition of this external drive.

> (FWIW here, while I do have backups, they aren't always current.  Still,
> for my purposes the *REAL* backups are the experiences and knowledge in
> my head.  As long as I have that, I can recreate the real valuable stuff,
> and to the extent that I can't, I don't consider it /that/ valuable.  And
> if I lose those REAL backups... well I won't have enough left then to
> realize what I've lost, will I?  That's ultimately the attitude I take,
> appreciating the real important stuff for what it is, and the rest, well,
> if it comes to it, I lose what I lose, but yes, I do still keep backups,
> actually multiple levels deep, tho as I said they aren't always current.)

Hehe, good philosophy :-) .

> However, one trick that I alluded to, that actually turned out to be an
> accidental side effect feature of fixing an entirely different problem,
> is setting mixed-blockgroup mode at mkfs.btrfs and selecting dup mode for
> both data and metadata at that time as well.  (In mixed-mode, data and
> metadata must be set the same, and the default except on ssd is then dup,
> but the point here is to ensure dup, not single.)  As I said, the reason
> mixed-mode is there is to deal with really small filesystems and it's the
> default for under a gig.  And there's definitely a performance cost as
> well as the double-space cost when using dup.  But it *DOES* allow one to
> run dup mode for both data and metadata, and some users are willing to
> pay its performance costs for the additional data integrity it offers.

That is an interesting idea.  I might consider that.  Or I might just create a
third partition and make a RAID 1 out of those, once I know how much space my
backups will ultimately take.

But really, why is there no dup for data?

(I only set up my backups about a month ago just before my migration to BTRFS,
using rsnapshot, and the backups aren't fully there yet; the one monthly backup
is still missing, and I wanted to wait a bit after that to see how much space
the backups ultimately require.  Plus, I might back up (parts of) my laptop to
there, too, although there isn't that much stuff on it that isn't already
synchronised in some other fashion, so it's not decided yet.)

> Certainly, if you can possibly do two devices, the paired device raid1
> mode is preferable, but for instance my netbook has only a single SATA
> port, so either mixed-bg and dup mode, or partitioning up and using two
> partitions to fake two devices for raid1 mode, are what I'm likely to do.
[...]

Ah, you mentioned the RAID 1 idea already :-) .

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup