[gentoo-desktop] Re: System problems - some progress

[Duncan]

Lindsay Haisley posted on Fri, 25 Mar 2011 07:48:12 -0500 as excerpted:

> Yeah, I missed it the first time around.
>
> I posted to linuxforums.org and referenced the thread to the Central TX
> LUG list, and Wayne Walker, one of our members, jumped on and saw it
> right away.  We have some very smart and Linux-savvy folks here in
> Austin!  The CTLUG tech list is really good.  We have IBM, AMD, Dell,
> and a bunch of other tech companies in the area and the level of Linux
> tech expertise here is exceptional.  You'd be welcome to join the list
> if you're interested.  See <http://www.ctlug.org>.  We have people on
> the list from all over the world.
>
> I expect the LVM system will come up now, and it only remains to be seen
> if I can get the legacy nVidia driver to build.

This might be a bit more of the "I don't want to hear it" stuff, which you 
can ignore if so, but for your /next/ system, consider the following, 
speaking from my own experience...

I ran LVM(2) on md-RAID here for awhile, but ultimately decided that the 
case of lvm on top of md-raid was too complex to get my head around well 
enough to be reasonably sure of recovery in the event of a problem.

Originally, I (thought I) needed lvm on top because md-raid didn't support 
partitioned-RAID all that well.  I migrated to that setup just after what 
was originally separate support for mdp, partitioned md-raid, was 
introduced, and the documentation for it was scarce indeed!  But md-raid's 
support for partitions has VASTLY improved, as has the documentation, with 
partitions now supported just fine on ordinary md-raid, making the 
separate mdp legacy.

So at some point I backed everything up and reorganized, cutting out the 
lvm.  Now I simply run partitioned md-raid.  Note that here, I have / on 
md-raid as well.  That was one of the problems with lvm, I could put / on 
md-raid and even have /boot on md-raid as long as it was RAID-1, but lvm 
requires userspace, so either / had to be managed separately, or I had to 
run an initrd/initramfs to manage the early userspace and do a pivot_root 
to my real / after lvm had brought it up.  I was doing the former, not 
putting / on lvm, but that defeated much of the purpose for me as now I 
was missing out on the flexibility of lvm for my / and root-backup 
partitions!

So especially now that partitioned md-raid is well supported and 
documented, you may wish to consider dropping the lvm layer, thus avoiding 
the complexity of having to recover both the md-raid and the lvm if 
something goes wrong, with the non-zero chance of admin-flubbing the 
recovery increasing dramatically due to the extra complexity of additional 
layers and having to keep straight which commands to run at each layer, in 
an emergency situation when you're already under pressure because things 
aren't working!

Here, I decided that extra layer was simply NOT worth the extra worry and 
hassle in a serious recovery scenario, and I'm glad I did.  I'm *FAR* more 
confident in my ability to recover from disaster now than I was before, 
because I can actually get my head around the whole, not just a step at a 
time, and thus am FAR less likely to screw things up with a stupid fat-
finger mistake.

But YMMV, as they say.  Given that the capacities of LVM2 have been 
improving as well (including its own RAID support, in some cases sharing 
code with md-raid), AND the fact that the device-mapper services (now part 
of the lvm2 package on the userspace side) used by lvm2 are now used by 
udisks and etc, the replacements for hal for removable disk detection and 
automounting, etc, switching to lvm exclusively instead of md-raid 
exclusively, is another option.  Of course lvm still requires userspace 
while md-raid doesn't, so it's a tradeoff of initr* if you put / on it 
too, vs using the same device-mapper technology for both lvm and udisks/
auto-mount.

There's a third choice as well, or soon will be, as the technology is 
available but still immature.  btrfs has built-in raid support (as with 
lvm2, sharing code at the kernel level, where it makes sense).  The two 
biggest advantages to btrfs are that (1) it's the designated successor to 
ext2/3/4 and will thus be EXTREMELY well supported when it matures, AND 
(2) because it's a filesystem as well, (2a) you're dealing with just the 
one (multi-faceted) technology, AND (2b) it knows what's valuable data and 
what's not, so recoveries are shorter because it doesn't have to deal with 
"empty" space, like md-raid does because it's on a layer of its own, not 
knowing what's valuable data and what's simply empty space.  The biggest 
disadvantage of course is that btrfs isn't yet mature.  In particular (1) 
the on-disk format isn't officially cast in stone yet (tho changes now are 
backward compatible, so you should have no trouble loading older btrfs 
with newer kernels, but might not be able to mount it with the older 
kernel once you do, if there was a change, AFAIK there have been two disk 
format changes so far, one with the no-old-kernels restriction, the latest 
without, as long as the filesystem was created with the older kernel), AND 
(2) as of now there's not yet a proper fsck.btrfs, tho that's currently 
very high priority and there very likely will be one within months, within 
a kernel or two, so likely available for 2.6.40.

Booting btrfs can be a problem currently as well.  As you may well know, 
grub-1 (0.9x) is officially legacy and hasn't had any official new 
features for years, /despite/ the fact that last I knew, grub2's on-disk 
format wasn't set in stone either.  However, being GPLv2-ed, the various 
distributions have been applying feature patches to bring it upto date for 
years, including a number of patches used routinely for ext2/3 
filesystems.  There is a grub-1 patch adding btrfs support, but I'm not 
sure whether it's in gentoo's version yet or not.  (Of course, that 
wouldn't be an issue for your new system as you mentioned it probably 
won't be gentoo-based anyway, but others will be reading this too.)

The newer grub-2 that many distributions are now using (despite the fact 
that it's still immature) has a btrfs patch as well.  However, there's an 
additional complication there as grub-2 is GPLv3, while the kernel and 
thus btrfs is GPLv2, specifically /without/ the "or later version" 
clause.  The existing grub-2 btrfs support patch is said to have worked 
around that thru reverse engineering, etc, but that is an issue for 
further updates, given that btrfs' on-disk-format is NOT yet declared 
final.  Surely the issue will eventually be resolved, but these are the 
sorts of "teething problems" that the immature btrfs is having, as it 
matures.

The other aspect of booting to btrfs that I've not yet seen covered in any 
detail is the extent to which "advanced" btrfs features such as built-in 
RAID and extensible sub-volumes will be boot-supported.  It's quite 
possible that only a quite basic and limited btrfs will be supported for 
/boot, with advanced features only supported from the kernel (thus on /) 
or even, possibly, userspace (thus not on / without an initr*).

Meanwhile, btrfs is already the default for some distributions despite all 
the issues.  How they can justify that even without a proper fsck.btrfs 
and without official on-disk format lock-down, among other things, I don't 
know, but anyway...  I believe at present, most of them are using 
something else (ext2 or even vfat) for /boot, tho, thus eliminating the 
grub/btrfs issues.

But I do believe 2011 is the year for btrfs, and by year-end (or say the 
first 2012 kernel, so a year from now, leaving a bit more wiggle room), 
the on-disk format will be nailed-down, a working fsck.btrfs will be 
available, and the boot problems solved to a large extent.  With those 
three issues gone, people will be starting the mass migration, altho 
conservative users will remain on ext3/ext4 for quite some time, just as 
many haven't yet adopted ext4, today.  (FWIW, I'm on reiserfs, and plan on 
staying there until I can move to btrfs and take advantage of both its 
tail-packing and built-in RAID support, so the ext3/ext4 thing doesn't 
affect me that much.)

That leaves the two choices for now now, md-raid and lvm2 including its 
raid features, with btrfs as a maturing third choice, likely reasonable by 
year-end.  Each has its strong and weak points and I'd recommend 
evaluating the three and making one's choice of /one/ of them.  By 
avoiding layering one on the other, significant complexity is avoided, 
simplifying both routine administration and disaster recovery, with the 
latter a BIG factor since it reduces by no small factor the chance of 
screwing things up /in/ that recovery.

Simply my experience-educated opinion.  YMMV, as they say.  And of course, 
it applies to new installations more than your current situation, but as 
you mentioned that you are planning such a new installation...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman