[gentoo-amd64] Re: grub and maximum kernel file size

[Duncan]

Branko Badrljica <brankob@××××××××××.com> posted
49DE848D.60905@××××××××××.com, excerpted below, on  Fri, 10 Apr 2009
01:28:13 +0200:

> Duncan wrote:
>
>
>> Did you try md-mod,start_dirty_degraded=1 (AFAIK this applies to
>> RAID-4/5/6 only)?
>>
> No

Hmm, and you said RAID-5, right?

I think one of the reasons I may have never had to use that option on 
RAID-6 is that it has two-way parity, so chances are either one or the 
other will still be valid -- even if it crashes in the middle of a write, 
one stripe-write might be invalid, but as long as I wasn't already 
running degraded, the state on the others should be clean, either clean-
state from before the write, or clean-state from after the write, thus 
ensuring data integrity.  Naturally, due to hardware cache buffers, etc, 
this isn't entirely guaranteed (unless the write buffers are disabled), 
but the chances are far higher of a clean state than they are with RAID-5 
with its single-way parity.

That's one reason that at least after I groked RAID-4/5/6 a bit better, 
I've been keen to get back a fully functional RAID-6 after a crash with 
one of the disks returning unclean on reboot, thus degrading the RAID 
until it is recovered.  In I believe two cases I've degraded to *NO* 
redundancy, dropping TWO disks out due to not catching the one dropped 
out for some time (this was before I really groked this RAID stuff, so I 
wasn't being quite as careful).  Had I crashed again before recovery of 
at least one of the drives in those cases, thus before getting back at 
least single-way-parity, chances are I'd have had to use the 
start_dirty_degraded option, with its corresponding risk of lost data 
(the reason for the option and its defaulting OFF) due to failure of the 
data integrity guarantee RAID-4/5/6 normally includes.

(FWIW I do run without UPS. Unfortunately the reality is that a proper 
UPS for this level of machine, at least back when I was running CRTs, 
would cost near what the machine itself did, and given that I was already 
buying near the top of my budget range that put a UPS out of 
consideration.  Since I switched to LCD I've been considering it again 
and would likely have one by now had the economy and therefore my budget 
not tanked.)

At least as I understand it, given RAID-5 only, loss of one spindle as 
"dirty" is fine, but if the system crashes while recovering it and the 
write-intent bits indicate an uncompleted write on the two remaining 
spindles, that option is needed as the alternative would be loss of the 
data integrity guarantee.  With a RAID-6, there's the additional 
protection of two-way parity and thus a second spindle can drop out 
without killing the data integrity guarantee, but the risk remains the 
same if there's a crash in that state and a /third/ drops out as unclean.

So I guess I'm glad I decided to go with the additional protection of 
RAID-6, even if I did so before I fully groked what I was doing.

>> What about listing the appropriate component devices, as so:
>> md=d1,/dev/sda1,/dev/sdb1,/dev/sdc1... ?
>>
> Yes. I had it by default. Without it never seemed to work. With it, it
> worked _sometimes_.

Wow.  I guess under those conditions I'd probably be going initramfs 
too.  Fortunately my experience has been far more robust, even without a 
UPS.  If that has been due to my choice of RAID-6, over a RAID-5 with a 
hot-spare, I guess I'm glad I made the cautious choice even if I didn't 
fully groke the implications back when I made it.
 
> Initramfs seem to be superior solution. With it there is more stuff I
> can do, like trying to rebuild array, fsck filesystems thoroughly etc.
>
> In fact, this is what it does now. Amongst other things, it checks
> /etc/fstab, gets filesystem types for "/boot" and "/" and e2fscks them,
> if needed.

That's what my "rootbak" partition is for. =:^)
 
> Being early adopter of ext4 I was many times in situation when boot-time
> e2fsck found some serious errors but demanded to be rerun on unmounted
> media to be able to finish. Without initramfs, only way to do this is
> through CD, but kernel there doesn't recognise disks on my Dell. Besides
> that it is tedious.

Well, you /could/ burn a CD with a kernel built with the proper options, 
but you are correct, booting from a CD is indeed a hassle.

But again, a "rootbak" copy of your root partition to fall back on would 
work just as well as the initramfs, and backups are something people 
should be doing anyway, so for me anyway, it was less (as in no 
additional) work.

> That thingie is _definitely_ very useful and I plan to make it
> permanent.

Indeed, it sounds like it.  Of course, a fully functional second copy of 
the root partition would be even more useful. =:^)

> I had completed the script that does the packing in maybe two hours ( I
> don't remeber anymore ) and it needs maybe a few minutes to make initrd,
> copy it to /boot/initrd and old one to /boot/initrd_old. Also, if I need
> one more package in the initrd ( say new version of gcc or python etc),
> it is only matter of adding package name in one line...
>
> But I will do something much more compact in C. With maybe an option to
> do "DefCon 4" style boot from USB key toolshed.
>
> I found USB keys to be useful for that role, when opened and with
> soldered "in-line" conenctor.  With it it becomes small dongle that is
> plugged directly into motherboard and so practically part of the
> machine.
>
> Tha way, you get "Terminator style" sturdy machine that is capable of
> reboot no matter what just as motherboard, CPU and PSU stay functional.
> ;o)

Indeed.  This sounds actually very practical.  I believe there's actually 
motherboards that do it, but it's a high-end feature, with a roll-your-
own being much more flexible and much cheaper, for those willing to do it.

Actually, I've seen mods for *-WRT class routers and software (OpenWRT, 
DD-WRT, Tomato, etc) that do pretty much the same thing, solder a USB 
based flash module to the thing and expand their storage and flexibility 
by an order or two of magnitude!  Run a server, etc, directly off your 
router!  They can be made detachable, or not, as desired.

> Yeah, but what do you do if your raid becomes non-functional for some
> reason ?
>
> I have four disks and two RAIDs. First is boot bartition RAID-1, which
> ebncompasess firs 1GB of each disk, second is RAID-5, which has the rest
> of the capacity.
>
> All partitions are typed as "fd"- self-detect RAID. On boot it doesn't
> matter. BIOS and GRUB see first partitions as if they were ordinary
> partitions with ext2 that happen to have exactly the same content.
>
> So, if one or more of the disks failed, I could boot with initrd from
> whatever disk is left...
>
> While in initramfs, I check and assemble both arrays and exec
> "make-init" ( or whatever its name it is- that util in klibc- it is
> effectively thingie that erases everything in /rootfs, remounts new root
> to / and starts init )

Here, it's three RAIDs on 4 disks/spindles, but all three covering the 
same four spindles, with one each RAID-0/1/6, each running in parallel 
partitions on each of the four spindles.

I had initially intended to put recovery software on the RAID-1, and 
sized it accordingly.  However, after I was setup and it was time to do 
so I decided I didn't actually need it.

But if I were doing it over, as I said earlier, I'd definitely create 
separate RAID-6s instead of the single partitioned RAID-6, and use two 
rootbaks so three root filesystem images total, one on each of three 
separate RAID-6s, so if one failed to come up, I'd just change the 
command line as necessary to bring up the second, and if it failed, the 
third.

I'd also split the RAID-1, either creating two single-disk RAID-1s or 
more likely, four separate /boot images direct on the individual spindles.

I don't think I'd do initramfs, but something that just occurred to me 
now -- I could actually copy the content of an entire rescue CD to one of 
the four disks (leaving three as /boot copies), setting it up as an 
alternative boot I could access directly from BIOS, to recover the rest 
of the system, should I need to.

>> I won't be doing nVidia any time soon.  Not until they come around to
>> properly cooperating with the FLOSS community.  YMMV and I know a lot
>> of gamers especially value their games above their freedom.  That's not
>> my life and not my systems and therefore not my call, but what I run
>> here IS, and if I wanted to be stuck with proprietaryware, I'd have not
>> bothered dumping a decade's worth of experience on MS to start over
>> with Linux!
>>
> Well, at least every fresh version of their driver is available on
> Gentoo at the same instant it gets out. Even without it, nvidia's own
> installer can figure things out.
>
> Not so with Radeon. I needed 9.1 version of driver when latest in gentoo
> tree was 8.1. I downloaded the thing and died wrestling with damn
> scripts halfway through.

The community wisdom has been this:

1) If you want an expansion card and freedomware drivers, go with (older) 
Radeons.  For many years it was r2xx based chips at the latest so 92xx 
series cards.  Now the r3xx-r5xx series, pretty much all the xXXX (x300-
x1900) cards, are quite well supported, including 3D.  The r600 and r700 
series, basically the hd* cards, aren't as well supported /just/ yet, but 
development is coming on strong since the specs were released and they 
look to be reasonably well supported by this time next year.

2) For the gamers out there and anyone else needing the features only to-
date available in proprietaryware only drivers due to lack of specs (or 
with the later Radeons, only recently available specs), nVidia's the way 
to go.  I believe it's also the choice for integrated video 
proprietaryware folks, tho I'm not as familiar with that market and 
honestly don't know.  ATI's proprietary drivers have been improving 
recently (since AMD took them over and they turned over a new leaf in 
regard to Linux), but for years were, plainly stated, crap.  While nVidia 
has refused to open the specs, they HAVE been reasonably good -- by far 
and away the best -- with their closed source Linux drivers, so there's 
no question, that's the way to go if you don't care about the 
proprietaryware aspect and are willing to jump thru the hoops and accept 
the delays it forces on users and the community in general.

3) For integrated use with freedomware drivers, in the historical mid-
term, Intel was the top choice and remains a good one.  Of course, that 
means Intel systems as well, AMD fans need not apply, but that's the way 
it is.  Since AMD bought ATI and with the resulting changes there, in 
theory they should have a working integrated video AMD/ATI platform 
freedomware solution either new or more likely coming, but I really don't 
know.  In the laptop/integrated market today, I'd personally chose Intel, 
no question about it.

> I don't get what kind of machine did they write that thing for- it sure
> as hell wasn't anything like mine.
> Also, when it comes to OpenGL performance, nVidia is still the king as I
> understand.

Yes.  As I said, ATI's proprietaryware Linux drivers have had the 
reputation as crap.  If you are doing proprietaryware, until /very/ 
recently (as in this year), nVidia, hands-down.  That said, ATI's latest 
have improved a lot, from what I've read, to the point Phoronix among 
others seems to be recommending them evenly with nVidia now.  /That/ 
said, I'm honestly not interested enough in the proprietaryware side to 
be the one to talk to about this.  But if you're in the market for brand 
new higher end graphics cards right now, do at least take a look at the 
available Phoronix.com resources.

> I have upgraded my dual setup to three-monitor setup ( 3x Samsung 204B,
> 3x1600x1200) and have had to add elcheapo 9400 card to existing dual-DVI
> 8800GT.
>
> Do you know about AMD/ATI Linux drivers- do they do such kind of stuff
> without extra wizardry ?

I don't know much about the proprietaryware side at all.  At present, I 
don't know that much about the newer hardware freedomware drivers either, 
tho I expect I'll know much more later this year as I'm just now starting 
to look at upgrading my aging Radeon 9200, since there's actually decent 
support for newer stuff now.  

What I've seen so far is as I said above, thru the r500 series, basically 
the Radeon xXXXX cards, support is now close to that on the older cards 
and since they're newer, that means better/more-capabilities in many 
cases.  But for the current r600 and r700 parts, basically hd* cards, 
support is under fast and furious development but is lagging six months 
minimum, more like nine.  Still, by next spring support should be quite 
good.

I expect I'll be buying a near top of the series r500 xXXXX card, here, 
for three reasons.  (1) Support is more solid currently.  (2) I'm still 
running AGP and PCI-X, NOT PCI-E (FWIW, this is the last major system 
upgade I plan on this thing, tho I expect to be running it several years 
still), so for me, the benefits (or even availability) of the r600/r700 
chip HD* cards aren't likely to be that great over the cheaper r500 chip 
xXXXX cards.  (3) I could never justify $400+ for a graphics card in any 
case, and as a couple generations out, r5xx chip xXXXX cards should be 
cheap in comparison to the r6xx/r7xx chip HD* cards.

Specifically to your question, I really don't know on the multi-card 
thing.  I had it working years ago (xf86 era, with the nVidia card and 
proprietary driver and an old 4 MB S3 Virge), but have stuck with single-
card dual-out solutions since upgrading to dual 21/22" CRTs @ 1600x1200 
each some years ago, and later to dual 1920x1200 LCDs, currently.  (I 
wanted the 30" 2560x1600 LCDs, but $1000+ each plus a new card at the 
same time, to handle dual-dual-link-DVI, wasn't in the cards.  There's 
always the next upgrade cycle, I guess.)  I /had/ tried it with early 
xorg era and Radeon r1xx/r2xx based cards and freedomware drivers, but 
didn't get it working.  I've not tried it since, however, as resolutions 
and screen sizes have improved to the point where I've not found it as 
pressing as it once was.

I read somewhere that support for multi-card had broken at one point in 
the transfer from the old multi-screen entry layout section
hard-xorg.conf-configured setups to the newer dynamic PnP xrandr based 
setups, but I don't know if that was xorg-wide, just for that driver, 
just for xrandr at that point, or what.  I also have little idea whether 
it has been fixed, altho obviously if you're running three monitors on 
two cards with an nVidia driver, it still works with that.

Anyway, ask me in Sept. or Nov. or so, and I'll probably be far more 
informed, as I'll be fresh off of doing all the research necessary to 
determine what I want for an upgrade, instead of just starting it. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman