Gentoo Archives: gentoo-amd64

From: Duncan <1i5t5.duncan@×××.net>
To: gentoo-amd64@l.g.o
Subject: [gentoo-amd64] Re: hp proliant 585 cciss grub problems and solving
Date: Wed, 22 Sep 2010 08:03:05
In Reply to: [gentoo-amd64] hp proliant 585 cciss grub problems and solving by Benny Pedersen
1 Benny Pedersen posted on Wed, 22 Sep 2010 03:30:35 +0200 as excerpted:
4 > with 2.6.35 there is problem to load firmware to tg3 ethernet, it works
5 > in 2.6.34
7 As I run direct Linus-tree git (tho I'm a couple rcs behind ATM), even
8 2.6.35 is somewhat back in history for me. But IIRC, somewhere in there
9 (tho I had thought it was 2.6.34??), they switched around the kernel
10 firmware loading options a bit, and a make oldconfig didn't really quite
11 do it in terms of getting the new settings working like the old ones did.
12 So that might be you're seeing. Get it setup correctly, however, and it
13 should again be "totally copacetic, man!" =:^)
15 FWIW, I have tg3 here and it works fine. Actually, at least my hardware
16 works fine even without the firmware -- all the firmware does is hardware
17 offloading of something or other, AFAIK, so it simply handles it in
18 software without it. But the kernel will complain about not having the
19 firmware if it can't find it, so loading it shuts up that complaint and as
20 mentioned, apparently does a bit more in the hardware/firmware instead of
21 on the main CPU, with it.
23 Anyway, here's how I'm configured here. (It's worth noting that I build-
24 in everything, so no modules to load or keep track of, firmware built-in
25 as well. FWIW, I'm a free software guy and won't run proprietary drivers,
26 but if it's simply ROM alternative firmware, I don't worry about it --
27 that's where I personally draw my line. So I don't worry about either the
28 tg3 or the radeon firmware I run.)
30 In menuconfig under Device Drivers, Generic Driver Options, Include in-
31 kernel firmware blobs in kernel binary, CONFIG_FIRMWARE_IN_KERNEL. (You
32 can probably do without this if you load tg3 as a module.). Also, in the
33 same place, Userspace firmware loading support CONFIG_FW_LOADER, altho
34 here, that's hard-configured on by some other option I have configured,
35 it's not something I can disable. I also have Firmware blobs root dir set
36 to /lib/firmware, tho I think that's the default. Finally, under normal
37 conditions, Prevent firmware from being built,
38 CONFIG_PREVENT_FIRMWARE_BUILD, is turned on here, but AFAIK I had to turn
39 it off temporarily to build the firmware once after things moved around,
40 then I could turn it on again.
42 The other piece of the puzzle, which didn't /used/ to apply to the tg3
43 firmware and I don't /believe/ it does yet, but it may well apply at some
44 point in the future, is that they're gradually moving firmware that
45 /used/ to be bundled with the kernel into a separate firmware package. In
46 Gentoo, this is the sys-kernel/linux-firmware package. If you can't find
47 the firmware file it's complaining about in either the kernel tree or the
48 /lib/firmware/ location, you may need to install that package.
50 > it leads me to the next question, is there not anyplace where a default
51 > kernel config exists for most big server platform, or does others just
52 > customize it so much there is not a common good one ?
54 I'm guessing most server folks, at least Gentoo server folks, customize
55 their kernel config. In particular, for publicly accessible servers,
56 building all modules into the kernel itself and disabling module loading
57 entirely is considered the secure way to do it, because it kills the
58 chance a cracker might otherwise have of loading a custom kernel module
59 with all his "toys". Since a no-module kernel is by definition a
60 customized kernel...
62 > any tips to share ?
64 I'm not working at that level and not dealing with a publicly accessible
65 server (nor with virtualization, unfortunately my hardware is pre-hardware-
66 virtualization-support =:^( ), but my hardware is now aging server-grade
67 from 2003/2004-ish (looks like yours is similar age, but you're running 8-
68 cores total while I'm running dual dual-cores, four-cores total, and I'm
69 only running 8 gig RAM compared to your 32), and I play around with kernel/
70 md-RAID and the like.
72 Some tips in general, tho. These aren't knew and you may well already
73 know them if you've been around for awhile.
75 1) Are you going to be running publicly accessible or not? At least at
76 the level the machine is public (thus at least virtualized if it's all VMs
77 that are public, not "domain 0", but the whole machine if not everything
78 publicly accessible is virtualized), the secure configuration is don't
79 have a build environment available. Build the system on another machine,
80 and only install (from binpkg, or better yet, rsync, and don't even have
81 portage on the public server) what's necessary to run the public servers
82 on the publicly accessible machines.
84 Of course, if it's not publicly accessible and all use is at least semi-
85 trusted, that's much less of an issue. If it's your workstation/play-
86 thing as is the case here (or if it's your dedicated build-server,
87 obviously), it's probably less trouble to do all your builds on the
88 machine itself, putting all that memory and CPU horsepower to work.
90 2) If you are using it as a build machine, and this really applies to
91 anything with 4 gig or more of RAM, pointing PORTAGE_TMPDIR at a tmpfs
92 mounted filesystem DRAMATICALLY speeds up your builds. General rule of
93 thumb is a minimum 4 gigs tmpfs (on 4 gig systems it can swap if
94 necessary, still tends to be faster than building on conventional spindle
95 backed filesystems) and at least a gig per core, so you'll want to set
96 your tmpfs max size to at least 8 gigs.
98 From my experience, the kernel doesn't get interactive-unresponsive from
99 just build-jobs alone, as long as you're not hitting the disk too hard.
100 So with for instance the very highly parallelizable kernel builds that use
101 rather less memory and scratch space per job than most compiles will, it's
102 possible to run a multi-hundred load-average even on a dual-core, and as
103 long as you're not swapping or otherwise hitting disk too hard, the system
104 remains actually reasonably responsive /especially/ when you consider it's
105 running a multi-hundred load average!
107 As a result, what I tend to do is set a reasonably high -j number (or
108 simply -j, unlimited), then use the -l load average limit, not to limit
109 load average itself, but as an indirect way to control the memory usage.
110 On my older dual-dual-core, with 6 gigs RAM (it's normally 8 but a stick
111 died and I've not replaced it) and with portage building in a 6-gig tmpfs,
112 I use MAKEOPTS="-j13 -l10", and use emerge --jobs=4 --load-average=7.
113 That causes make to try to trigger more jobs if it can to try to keep a
114 load average of 10, but if it can't and the load-average falls below 7,
115 portage will start merging another package in parallel (if the dependency
116 tree will let it do so), with upto 4 packages merging at once.
118 IIRC with the full 8 gigs, I ran something like -j20 -l16 MAKEOPTS (so a
119 load of four per core), with emerge --jobs=10 --load-average=12. With a
120 full 32-gig RAM and 8 cores, you should be able to comfortably double
121 that, tho I'd play around with it a bit, as it'll depend on your hardware
122 to some extent. You /probably/ don't want to go /too/ much above that,
123 not because your system shouldn't handle it (it should), but because at
124 some point, the system starts spending more time switching between jobs as
125 opposed to actually doing them. Once you're running near 100% most of the
126 time, additional jobs are likely to actually slow things down. I'm not
127 positive of this, but given what I understand of Intel hardware of that
128 age, it actually didn't parallel as well as AMD hardware did, due to
129 memory bottlenecks, etc, that AMD's on-board memory controllers avoided.
130 So certainly, do some experimenting, but with 32-gig memory and 8 cores,
131 something like -j40 -l32 and emerge --jobs=10 --load-average=20, should be
132 a good ballpark to start with.
134 (Note that in my experience, portage rarely paralleled more than five
135 package emerges at once in any case, and I think the most I've seen was
136 seven, no matter how high the --jobs and --load-average is set. It may
137 actually have gotten a bit better at it now, as the parallel merging
138 functionality was pretty new back when I was experimenting with that, but
139 at least back then, it seemed to be some built-in limitation on its
140 dependency resolver -- it just wouldn't start more no matter how many I
141 gave it permission to parallel. So that's why I didn't up the --jobs=
142 beyond 10, it didn't seem to make an difference anyway, at least here.
143 Perhaps with your more-cores hardware and a newer portage, it'll actually
144 run more than 5-7 parallel merges at once, given the chance, but I was
145 never able to get it to.)
147 Finally, you probably want PORTAGE_NICENESS=19 as well. Not only does
148 that lower the build priority to idle, so other tasks including
149 interactive responsiveness takes priority, but at nice=19, the kernel
150 automatically puts that task in batch-scheduling mode as well, giving it
151 longer timeslices, so it can get more done in each one with less task-
152 switching overhead. Thus, on an otherwise lightly loaded system, it's
153 actually quite possible that a heavy CPU cycles oriented task like
154 building packages will complete faster at nice=19 than it would at nice=0,
155 because at nice=19, the overhead from task-switching goes down.
157 I'll throw in one more as well, tho this has little or nothing to do with
158 servers, but applies to pretty much everyone. One of my favorite under-
159 advertised portage features is FEATURES=buildpkg. To enable it you'll
160 want to allow at least 4 gigs of package space for it -- I have my PKGDIR
161 set to its own dedicated 4-gig partition and I can let it accumulate about
162 two complete package sets, more of many packages, before I start running
163 low on room, but if you've hundreds of gigs of disk space without anything
164 to use it for, allowing 6 or 8 gigs would allow you to forget about it for
165 quite some time (8 gigs will likely be over a year's worth of packages)
166 before worrying about cleaning it out. The binpkgs this produces have
167 come in handy many times, here. Among other things, they can be useful if
168 your gcc ever breaks, because you can still install an older version from
169 binpkg. And because the binpkg format is simply a bzip2ed tarball with
170 some extra metadata tacked on the end, you can browse them with your
171 favorite archiver as well. This allows you to check what files the
172 package contained two versions ago, for instance, or compare the default
173 config files between versions or against what's installed on your live
174 system. As a last resort, it's also possible to repair broken portage or
175 python installations by simply untarring the package directly over the
176 live filesystem (best to backup config files first, as that'll bypass
177 portages config-protect system, etc), bypassing the broken portage
178 entirely in ordered to get a working system once again. (If you do this,
179 once you do get a working portage and have replaced the default configs
180 with the versions you backed up once again, the first thing to do is
181 remerge the working binpkg back over itself, since you bypassed portage
182 with the untar, so it knows nothing about it and still thinks you have the
183 broken version installed. This will sync portage's installed package
184 database with what's on the disk once again. As mentioned, this is a last
185 resort /because/ of the config file and out-of-sync portage database
186 issues, but if you have a broken portage and are otherwise stuck, it can
187 save your behind, as it has mine, a couple times.)
189 --
190 Duncan - List replies preferred. No HTML msgs.
191 "Every nonfree program has a lord, a master --
192 and if you use the program, he is your master." Richard Stallman


Subject Author
Re: [gentoo-amd64] Re: hp proliant 585 cciss grub problems and solving Benny Pedersen <me@××××.org>