Re: [gentoo-hardened] kernel upgrade problems: bad page state

[Brian Kroth]

Natanael Copa wrote:
> On Sun, 2007-11-04 at 01:34 +0200, pageexec@××××××××.hu wrote:
>> On 3 Nov 2007 at 19:02, Brian Kroth wrote:
>
>>> PS - would you like me to take this off list?
>> no need unless you don't want to leak any more of your host names
>> in kernel logs ;-). actually, attachments this big you can send me
>> directly and don't bother the list with it, but the discussion may
>> be interesting for others as well.
>>
>
> This discussion is interesting, yes
>
> -nc
>

Getting more interesting.  It just rebooted itself, which it hasn't done 
yet.  Here's the two messages in kern.log before and after it happened. 
  Nothing unexpected in any of the other log files either.

Nov  5 09:58:42 tux-mc PAX: 19300 14a1edf0 1200e000-15000000 0001200e
Nov  5 11:14:02 tux-mc Linux version 2.6.23-hardened-r1 (root@tux-mc) 
(gcc version 3.4.6 (Gentoo Hardened 3.4.6-r2, ssp-3.4.6-1.0, 
pie-8.7.10)) #3 SMP Sun Nov 4 13:01:03 CST 2007

Here's what I've gotten on startup since then:

Nov  5 11:14:31 tux-mc PAX: 6190 11e13d40 11de2000-11e58000 00011de2
Nov  5 11:14:32 tux-mc PAX: 6274 4abafeb0 4abaf000-4abb1000 00000002
Nov  5 11:14:34 tux-mc IPMI Watchdog: driver initialized
Nov  5 11:14:56 tux-mc PAX: 5200 16d4796c 16d38000-16d64000 00016d38
Nov  5 11:15:20 tux-mc BUG: unable to handle kernel NULL pointer 
dereference at virtual address 00000c4a
Nov  5 11:15:20 tux-mc printing eip:
Nov  5 11:15:20 tux-mc c06ee937
Nov  5 11:15:20 tux-mc *pde = 00000000
Nov  5 11:15:20 tux-mc Oops: 0002 [#1]
Nov  5 11:15:20 tux-mc SMP DEBUG_PAGEALLOC
Nov  5 11:15:20 tux-mc Modules linked in: ipmi_watchdog bonding ipmi_si 
ipmi_devintf ipmi_msghandler e1000
Nov  5 11:15:20 tux-mc CPU:    1
Nov  5 11:15:20 tux-mc EIP:    0060:[<c06ee937>]    Not tainted VLI
Nov  5 11:15:20 tux-mc EFLAGS: 00255692   (2.6.23-hardened-r1 #3)
Nov  5 11:15:20 tux-mc EIP is at __down+0xcd/0xdb
Nov  5 11:15:20 tux-mc eax: 00000c4a   ebx: c0419295   ecx: f7add000 
edx: f1571500
Nov  5 11:15:20 tux-mc esi: 57c7b46d   edi: 00000c4a   ebp: f7f51ed8 
esp: f7f51ea8
Nov  5 11:15:20 tux-mc ds: 0068   es: 0068   fs: 00d8  gs: 0000  ss: 0068
Nov  5 11:15:20 tux-mc Process perl (pid: 6953, ti=f7f50000 
task=f5fba000 task.ti=d299c000)
Nov  5 11:15:20 tux-mc Stack: 00000c4a 00000000 00000f85 f7add030 
c180bd00 f7f51ef8 c0415117 00000c4a
Nov  5 11:15:20 tux-mc 00000000 c180bd84 00000000 50a3f400 00000408 
f7f51ef8 c053bf61 00000408
Nov  5 11:15:20 tux-mc 00000000 c180bd20 c180bd20 c180bd00 f7f51f20 
c04152bd 000220f2 00000000
Nov  5 11:15:20 tux-mc Call Trace:
Nov  5 11:15:20 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:15:20 tux-mc [<c0403f4b>] show_stack_log_lvl+0x8d/0xaa
Nov  5 11:15:20 tux-mc [<c04041ca>] show_registers+0x212/0x378
Nov  5 11:15:20 tux-mc [<c04044d6>] die+0x121/0x208
Nov  5 11:15:20 tux-mc [<c06f06f5>] do_page_fault+0x2dd/0x80a
Nov  5 11:15:20 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:15:20 tux-mc [<f7f51ef8>] 0xf7f51ef8
Nov  5 11:15:20 tux-mc =======================
Nov  5 11:15:20 tux-mc Code: 89 50 04 89 02 89 f0 ba 03 00 00 00 c7 45 
e0 00 01 10 00 c7 45 e4 00 02 20 00 e8 4b a9 d2 ff 89 f0 89 fa e8 af 02 
00 00 8b 45 d0 <c7> 00 00 00 00 00 83 c4 24 5b 5e 5f 5d c3 55 b9 05 00 
00 00 89
Nov  5 11:15:20 tux-mc EIP: [<c06ee937>] __down+0xcd/0xdb SS:ESP 
0068:f7f51ea8
Nov  5 11:15:39 tux-mc PAX: 7110 500cf1d0 500cf000-500d0000 00000015
Nov  5 11:16:21 tux-mc PAX: 7307 13565f30 134ae000-13b89000 000134ae
Nov  5 11:20:01 tux-mc PAX: 8336 51626edc 51626000-51628000 00000004
Nov  5 11:20:10 tux-mc PAX: 8719 55ee5d88 55ee4000-55ee7000 0000012d
Nov  5 11:25:57 tux-mc PAX: 11103 500cf1d0 500cf000-500d0000 00000015
Nov  5 11:30:10 tux-mc PAX: 12755 53b06e38 53300000-53b65000 00053300
Nov  5 11:30:10 tux-mc PAX: 12753 53b06990 53300000-53b65000 00053300
Nov  5 11:30:10 tux-mc Bad page state in process 'cactid'
Nov  5 11:30:10 tux-mc page:c104e3a0 flags:0x40000001 mapping:00000000 
mapcount:0 count:0
Nov  5 11:30:10 tux-mc Trying to fix it up, but a reboot is needed
Nov  5 11:30:10 tux-mc Backtrace:
Nov  5 11:30:10 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:30:10 tux-mc [<c0403ebc>] show_trace+0x12/0x14
Nov  5 11:30:10 tux-mc [<c0403fb6>] dump_stack+0x16/0x18
Nov  5 11:30:10 tux-mc [<c044bbd5>] bad_page+0x66/0x97
Nov  5 11:30:10 tux-mc [<c044c5c7>] free_hot_cold_page+0x71/0x176
Nov  5 11:30:10 tux-mc [<c044c6d6>] free_hot_page+0xa/0xc
Nov  5 11:30:10 tux-mc [<c044f18c>] __page_cache_release+0x84/0x90
Nov  5 11:30:10 tux-mc [<c044f1f8>] put_page+0x27/0x36
Nov  5 11:30:10 tux-mc [<c0455da8>] do_wp_page+0x235/0x48f
Nov  5 11:30:10 tux-mc [<c0457233>] handle_mm_fault+0x2bc/0x2ef
Nov  5 11:30:10 tux-mc [<c06f0764>] do_page_fault+0x34c/0x80a
Nov  5 11:30:10 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:30:10 tux-mc =======================
Nov  5 11:35:43 tux-mc PAX: 5743 5da97f48 5da5d000-5da9f000 000bffbe
Nov  5 11:40:37 tux-mc PAX: 5744 134ae718 134ae000-13baa000 000134ae
Nov  5 11:40:37 tux-mc PAX: 16212 134ae300 134ae000-13baa000 000134ae
Nov  5 11:40:37 tux-mc Bad page state in process 'nagios'
Nov  5 11:40:37 tux-mc page:c1084060 flags:0x40000001 mapping:00000000 
mapcount:0 count:0
Nov  5 11:40:37 tux-mc Trying to fix it up, but a reboot is needed
Nov  5 11:40:37 tux-mc Backtrace:
Nov  5 11:40:37 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:40:37 tux-mc [<c0403ebc>] show_trace+0x12/0x14
Nov  5 11:40:37 tux-mc [<c0403fb6>] dump_stack+0x16/0x18
Nov  5 11:40:37 tux-mc [<c044bbd5>] bad_page+0x66/0x97
Nov  5 11:40:37 tux-mc [<c044c5c7>] free_hot_cold_page+0x71/0x176
Nov  5 11:40:37 tux-mc [<c044c6d6>] free_hot_page+0xa/0xc
Nov  5 11:40:37 tux-mc [<c044f18c>] __page_cache_release+0x84/0x90
Nov  5 11:40:37 tux-mc [<c044f1f8>] put_page+0x27/0x36
Nov  5 11:40:37 tux-mc [<c0455da8>] do_wp_page+0x235/0x48f
Nov  5 11:40:37 tux-mc [<c0457233>] handle_mm_fault+0x2bc/0x2ef
Nov  5 11:40:37 tux-mc [<c06f0764>] do_page_fault+0x34c/0x80a
Nov  5 11:40:37 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:40:37 tux-mc =======================