[gentoo-amd64] Re: grub and maximum kernel file size

[Duncan]

Branko Badrljica <brankob@××××××××××.com> posted
49DFDE56.4000107@××××××××××.com, excerpted below, on  Sat, 11 Apr 2009
02:03:34 +0200:

> Duncan wrote:
>
>
>>> Duncan wrote:
>>>
>>>
>>>
>>>> Did you try md-mod,start_dirty_degraded=1 (AFAIK this applies to
>>>> RAID-4/5/6 only)?
>>>>
>>> No
>>>
>> Hmm, and you said RAID-5, right?

> BTW: Where is this mod-module name way of setting inbuilt module
> variables documented ?
> I have uvesafb built-in as part of kernel and had to check the sources (
> documentation was out of date at some point ) to coem up with command
> line setting, like "uvesafb.mode_option=1280x1024-16@60 uvesafb.mtrr=3".
>
> Could I also write "md.start_dirty_degraded=1" ?

That info came from the kernel Documentation/md.txt file (so normally 
/usr/src/linux/Documentation/md.txt ).  The first several sections talk 
about boot time assembly and detection of arrays, with the various kernel 
command line options.  That particular info is covered in the "Boot time 
assembly of degraded/dirty arrays" section, starting on line 54.

There's also a much more general kernel parameter list in 
Documentation/kernel-parameters.txt , and some x86_64 specific options in 
Documentation/x86/x86_64/boot-options.txt .  The documentation for 
individual modules may also contain additional command line options for 
them.

I have no idea whether simply md. instead of mod-md. would work or not.  
As I said, I've never actually had to use that one, and simply copied it 
verbatim from the documentation, first into my grub.conf, then later 
moved from there to an informational/reminder file I keep around in 
/boot/grub/ , viewable using grub's cat command.

>> I think one of the reasons I may have never had to use that option on
>> RAID-6 is that it has two-way parity, so chances are either one or the
>> other will still be valid

> Yes, but RAID-6 isn't simple expansion of RAID-5 concept.
>
> It has two parity chunks-P and Q. While P is simple XOR parity from
> RAID-5, Q is substantially compute intensive. Further, while Q
> generation is "expensive", CPU can utilise its SSE artillery and soften
> the perceived extra load.
>
> But when CPU has to actually use Q parity for data regeneration, things
> come to a crawl. That part is quite intensive and SSE is useless there,
> so best CPU can do is use tabelaric approach and regenerate data
> byte-by-byte.
>
> So SW RAID-6 has come to be a bad idea, at least for me and my
> applications. HW RAID-6 with good HW is another story altogether,
> though.
>
> With SW, you pay extra drive, small extra power consumption, some extra
> cycles for Q generation ( not quite inconsequiential) and BIG penalty
> for regeneration - so big that usually "doing bussines as usual" while
> in such degraded mode is out of the question.
>
> So after some experience with SW and HW RAID-6 i opted for RAID-5 with
> cold spare and regular backups for pretty much everything.

This Q information I didn't know, tho it doesn't particularly surprise 
me.  However, I believe the additional data integrity is worth it for 
me.  After all, if more than a single disk dropped out on a RAID-5, you'd 
be down for recreate and restore from backup anyway, so while RAID-6 may 
be slow in that circumstance it's certainly not worse than the 
alternative.

> I did run for some time all of my machines through UPS only to find this
> costs too much.
>
> So I switched to DC. I rectified all three 220VAC phases to  260-300V DC
> and backed this up by 20x 100Ah-12V lead-acid akus in series. Even
> without akus it feels pretty much close to having UPS. When interruption
> happens, 90% of the time it affects only one phase, so remaining ones
> take over and I don't feel a thing...

That makes sense -- you more or less rolled your own UPS.

But I'be be leary of working with that 260+ DC.  That's dangerous to be 
working with.  Unlike AC, if you touch DC, it grabs you and doesn't let 
go.

Now what I might do is run off of 12 VDC or the comms standard 48 VDC, 
with individual power supplies running off of that accordingly.

> For my remote location I can't have 20 akus there, so I am working on
> PSU modification which would allow me to work on two...

That's more like it... or four, for the 48 VDC I mentioned above.  If I'm 
not mistaken there's standard power supplies available that run off of 
that already, since it's comms standard (at least here in the US).

>> That's what my "rootbak" partition is for. =:^)
>>
> Yes. And I use same thing on my workstation. Still, that doesn't quite
> match robustness of initrd.
> With initrd, if you can get to the phase where grub takes over, you can
> pretty much boot. Even if your kernel can't work with root drive for
> whatever reason ( missing driver for SATA,AHCI, ext4 ...)

Well, that only works if you've compiled the missing drivers as modules 
and have them on the initramfs/initrd.  Then, yes, it works, but doing 
that adds complexity once again.

Another way of doing the same thing would be to simply build a second 
kernel, with all those extra drivers built-in.  That'd avoid the 
initramfs, while allowing you to run the normal kernel, thus avoiding the 
extra memory used by all those additional drivers, most of the time.  (A 
script to play switcheroo with the .config and run make oldconfig, 
keeping a normal and a backup/everything kernel config around, would be 
trivial.)  You'd only have to build the "everything" one perhaps every 
six months, or say every 2-3 mainline kernel releases (2.6.x), to keep it 
working with the userspace (udev, etc).

>> Well, you /could/ burn a CD with a kernel built with the proper
>> options, but you are correct, booting from a CD is indeed a hassle.
>>
> On Gentoo, how to do this is far from obvious.

I don't think it'd be too hard, tho it would take a bit of research.  
Catalyst (what they use to generate the Gentoo install media) may help.

But the other possibility is to just take a normal LiveCD distribution, 
probably one designed for rescue operations (SysRescue seems to come well 
recommended), and if necessary, replace its kernel with one including the 
specific options necessary to support your hardware.  That's simply a 
matter of mounting the ISO using the loopback device, changing out the 
(or adding an additional) kernel, adding a boot menu entry for it if 
desired, unounting the loopback, then burning the ISO to disk.

Or as you mentioned, use a thumb drive and do about the same thing.

>> Actually, I've seen mods for *-WRT class routers and software (OpenWRT,
>> DD-WRT, Tomato, etc) that do pretty much the same thing, solder a USB
>> based flash module to the thing and expand their storage and
>> flexibility by an order or two of magnitude!  Run a server, etc,
>> directly off your router!  They can be made detachable, or not, as
>> desired.
>>
> Cool. If you have a link to such a thing, it would be apppreciated ;o)

The one I saw was one of the mod-projects I found while researching 
OpenWRT, which I eventually decided was what I wanted on my Linksys 
WRT54GL (bought specifically TO run community firmware in, tho I wasn't 
sure which distribution I was going to use when I bought it), tho I've 
not actually modded the hardware yet.

(Actually, while OpenWRT is decent, I'm thinking about checking about 
cross-compiling Gentoo/MIPS for it.  As I looked around at the OpenWRT 
init system and etc, I realized that while I could hack it, it was setup 
much closer to say Red Hat or possibly Debian than Gentoo, and I was 
having Mandrake flashbacks.  There's nothing wrong with that at all, but 
I realized that if I were to really start hacking on it, having Gentoo on 
it so I was working with something currently familiar to me would very 
likely be about as much trouble as learning all the details of its init 
system, and running Gentoo on it would be MUCH MUCH cooler! =:^)  I 
suspect there may be a few Gentooers that have already done it, actually, 
but I've done very little research on that, yet.  Linux and computing is 
my hobby, so I'm not /too/ worried if I never get it done, but it's nice 
to have the idea available to play with, and who knows, maybe I /will/ 
get it done some day.)

Anyway, here's the OpenWRT site, let's see if I can find the info on that 
mod once again:

http://openwrt.org/

Here's the WIKI entry for single or dual serial port addition (watch the 
wrap):

http://oldwiki.openwrt.org/OpenWrtDocs(2f)
KamikazeHowto.html#head-17977c9dcc901d82265a1c843e70b40694cd8eb2

Here's a table of peripherals they've tested (USB, Bluetooth, etc):
http://oldwiki.openwrt.org/OpenWrtDocs(2f)TableOfPeripherals.html
and
http://oldwiki.openwrt.org/TableOfPeripheralHardware.html

The above mentions USB connected printers, thumb-drives, scanners, input 
devices, audio/VoIP/telephony, cameras, all sorts of stuff.  Some work 
well, some don't.  Also mentioned is SD/MMC direct-connect to the GPIO 
lines on the unit, tho it cautions IO is slow compared to USB access.

Here's the BIG list of all the routers, etc, OpenWRT supports, along with 
model-specific notes:

http://oldwiki.openwrt.org/OpenWrtDocs(2f)Hardware.html

OK, here's the modding page.
http://oldwiki.openwrt.org/OpenWrtDocs(2f)Customizing.html

And here's what I had in mind when I wrote the bit that got this whole 
thing started -- it wasn't USB, it was MMC.  See the picture (2nd/3rd 
links direct to images).  The USB mentioned above was for routers other 
than the Linksys ones.  Some have it already, others it can be added, but 
I don't see anything about adding it directly to the Linksys WRT54Gx 
series, only the SD/MMC stuff.

http://oldwiki.openwrt.org/OpenWrtDocs(2f)Customizing(2f)Hardware(2f)
MMC.html

http://oldwiki.openwrt.org/attachments/OpenWrtDocs(2f)Customizing(2f)
Hardware(2f)MMC/kiel.kool.dk-AllSolderingDone.jpg

http://oldwiki.openwrt.org/attachments/OpenWrtDocs(2f)Customizing(2f)
Hardware(2f)MMC/kiel.kool.dk-Reuter_complete.jpg

There's way more instructions and photos at that first link.

>> But if I were doing it over, as I said earlier, I'd definitely create
>> separate RAID-6s instead of the single partitioned RAID-6, and use two
>> rootbaks so three root filesystem images total, one on each of three
>> separate RAID-6s, so if one failed to come up, I'd just change the
>> command line as necessary to bring up the second, and if it failed, the
>> third.
>>
> Not every application has plenty of storage space.

While you are correct, given the size of hard drives these days and the 
fact that we ARE talking minimum four-spindle RAID (for the RAID-6 
mentioned), by the time you get two spindles or more worth of data 
stripes, space isn't generally that much of an issue.  Especially so when 
my root filesystem is < 5 gigs, and that's including both KDE 4 and KDE 
5.  Thus, the above suggested three RAID-6 root filesystem images, a main/
working and two backups, is only 15 gigs or so, four spindles each with 
three 2.5 gig partitions, each of which forms a stripe of its own RAID-6  
2.5 gig partitions are tiny on todays disks.  Leave a bit more room and 
make each partition five gigs, for 10 gigs of data stripe per RAID-6, and 
it's still only 15 gigs of partitions on each drive.

>
>> I'd also split the RAID-1, either creating two single-disk RAID-1s or
>> more likely, four separate /boot images direct on the individual
>> spindles.
>>
>>
> This is what I do now. Four way RAID-1 for boot is exactly that. GRUB
> sees it as four separate drives with ext2 that can be mounted exactly
> the same as any other drive, so I can boot from any of them. When kernel
> takes over, it assembles them into an md array, so you can mount /boot
> and update all four drives with exactly the same data, when needed...

Well, I'm running 4-way RAID-1 for /boot too, and yes, GRUB does see it 
as four separate drives.  However, the problem I've run into and why I 
would make it two separate RAID-1s or four separate individual 
partitions, one per spindle, is that if something (including my own fat-
fingering) writes a bad config to or deletes grub-stage-two from that 
RAID-1, it blows away ALL FOUR config/boot spindles AT ONCE.  It'd be 
MUCH better if I had it at least split in half, with a working and 
backup /boot, just as I do most of the rest of the system.

But as I was writing the last reply, I thought about doing it slightly 
differently.  Setup two or three of the individual partitions I'm using 
now for the RAID-1 as individual /boot and bootbak partitions, then setup 
one (or perhaps two) with a LiveCD/thumbdrive based distribution, say a 
rescue disk, modified to boot from that partition instead of from the CD 
or thumbdrive.  The effect would be similar to having a bootable rescue 
disk (SysRescue or whatever) directly on the hard drive, instead of 
having to dig around for the CD.  Should it come to that, I could use the 
BIOS boot selector to select that drive, and it would boot the SysRescue 
or whatever image, and I'd go from there.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman