Hide Forgot
Description of problem: ... SysRq : Trigger a crashdump Linux version 2.6.18-267.el5PAE (mockbuild.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Wed Jun 8 15:26:51 EDT 2011 BIOS-provided physical RAM map: BIOS-e820: 0000000000010000 - 00000000000a0000 (usable) BIOS-e820: 0000000000100000 - 000000003ffe0000 (usable) BIOS-e820: 000000003ffe0000 - 000000003ffefc00 (ACPI data) BIOS-e820: 000000003ffefc00 - 000000003ffff000 (reserved) BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved) BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved) BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved) user-defined physical RAM map: user: 0000000000000000 - 00000000000a0000 (usable) user: 0000000002000000 - 0000000009f5b000 (usable) 0MB HIGHMEM available. 159MB LOWMEM available. found SMP MP-table at 000fe710 Memory for crash kernel (0x0 to 0x0) notwithin permissible range disabling kdump Using x86 segment limits to approximate NX protection DMI 2.3 present. Using APIC driver default ACPI: PM-Timer IO Port: 0x808 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) Processor #0 6:11 APIC version 17 ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) Processor #1 6:11 APIC version 17 WARNING: maxcpus limit of 1 reached. Processor ignored. ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-15 ACPI: IOAPIC (id[0x03] address[0xfec01000] gsi_base[16]) IOAPIC[1]: apic_id 3, version 17, address 0xfec01000, GSI 16-31 ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high edge) Enabling APIC mode: Flat. Using 2 I/O APICs Using ACPI (MADT) for SMP configuration information Allocating PCI resources starting at 10000000 (gap: 09f5b000:f60a5000) Detected 1396.478 MHz processor. Built 1 zonelists. Total pages: 40795 Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=ttyS0,115200 irqpoll maxcpus=1 reset_devices hda=cdrom memmap=exactmap memmap=640K@0K memmap=130412K@32768K elfcorehdr=163180K Misrouted IRQ fixup and polling support enabled This may significantly impact system performance ide_setup: hda=cdrom Enabling fast FPU save and restore... done. Enabling unmasked SIMD FPU exception support... done. Initializing CPU#0 CPU 0 irqstacks, hard=c236e000 soft=c234e000 PID hash table entries: 1024 (order: 10, 4096 bytes) irq 209, desc: c2306700, depth: 1, count: 0, unhandled: 0 ->handle_irq(): c204e0ea, handle_bad_irq+0x0/0x1a6 ->chip(): c22972e0, 0xc22972e0 ->action(): 00000000 IRQ_DISABLED set unexpected IRQ trap at vector d1 Console: colour VGA+ 80x25 Dentry cache hash table entries: 32768 (order: 5, 131072 bytes) Inode-cache hash table entries: 16384 (order: 4, 65536 bytes) Memory: 117976k/163180k available (2192k kernel code, 13012k reserved, 913k data, 232k init, 0k highmem) Checking if this processor honours the WP bit even in supervisor mode... Ok. Calibrating delay loop (skipped), value calculated using timer frequency.. 2792.95 BogoMIPS (lpj=1396478) Security Framework v1.0.0 initialized SELinux: Initializing. selinux_register_security: Registering secondary module capability Capability LSM initialized as secondary Mount-cache hash table entries: 512 CPU: L1 I cache: 16K, L1 D cache: 16K CPU: L2 cache: 512K Intel machine check architecture supported. Intel machine check reporting enabled on CPU#0. Checking 'hlt' instruction... OK. SMP alternatives: switching to UP code Freeing SMP alternatives: 14k freed ACPI: Core revision 20060707 CPU0: Intel(R) Pentium(R) III CPU family 1400MHz stepping 01 Total of 1 processors activated (2792.95 BogoMIPS). ENABLING IO-APIC IRQs ..TIMER: vector=0x31 apic1=0 pin1=0 apic2=-1 pin2=-1 Using local APIC timer interrupts. Brought up 1 CPUs checking if image is initramfs... it is Freeing initrd memory: 6876k freed NET: Registered protocol family 16 ACPI: bus type pci registered PCI: PCI BIOS revision 2.10 entry at 0xfc6ae, last bus=3 PCI: Using configuration type 1 Setting up standard PCI resources ACPI: Interpreter enabled ACPI: Using IOAPIC for interrupt routing ACPI: No dock devices found. ACPI: PCI Root Bridge [PCI0] (0000:00) PCI: Ignoring BAR0-3 of IDE controller 0000:00:0f.1 ACPI: PCI Root Bridge [PCI1] (0000:01) ACPI: PCI Root Bridge [PCI2] (0000:03) ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKI] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKJ] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKK] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKL] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKM] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKN] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKO] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LNKP] (IRQs 3 4 5 6 7 9 11 12 14) *0, disabled. ACPI: PCI Interrupt Link [LUSB] (IRQs 3 4 5 6 7 *11 12 14) Linux Plug and Play Support v0.97 (c) Adam Belay pnp: PnP ACPI init pnp: PnP ACPI: found 11 devices usbcore: registered new driver usbfs usbcore: registered new driver hub PCI: Using ACPI for IRQ routing PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default pnp: 00:07: ioport range 0x800-0x89f could not be reserved pnp: 00:07: ioport range 0x8a0-0x8af has been reserved pnp: 00:07: ioport range 0x8c0-0x8c3 has been reserved pnp: 00:07: ioport range 0xc00-0xcd7 has been reserved pnp: 00:07: ioport range 0xf50-0xf58 has been reserved pnp: 00:0a: iomem range 0x0-0x9ffff could not be reserved pnp: 00:0a: iomem range 0x100000-0x3fffffff could not be reserved pnp: 00:0a: iomem range 0xf0000-0xfffff could not be reserved PCI: Bridge: 0000:01:08.0 IO window: disabled. MEM window: disabled. PREFETCH window: disabled. NET: Registered protocol family 2 IP route cache hash table entries: 2048 (order: 1, 8192 bytes) TCP established hash table entries: 8192 (order: 4, 65536 bytes) TCP bind hash table entries: 4096 (order: 3, 32768 bytes) TCP: Hash tables configured (established 8192 bind 4096) TCP reno registered apm: BIOS not found. audit: initializing netlink socket (disabled) type=2000 audit(1307932964.487:1): initialized Total HugeTLB memory allocated, 0 VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) Initializing Cryptographic API alg: No test for crc32c (crc32c-generic) ksign: Installing public key data Loading keyring - Added public key 89E8880D98539482 - User ID: Red Hat, Inc. (Kernel Module GPG key) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) pci_hotplug: PCI Hot Plug PCI Core version: 0.5 BIOS reported wrong ACPI idfor the processor ACPI Exception (evxface-0545): AE_NOT_EXIST, Removing notify handler [20060707] Real Time Clock Driver v1.12ac Non-volatile memory driver v1.2 Linux agpgart interface v0.101 (c) Dave Jones agpgart: unable to determine aperture size. agpgart: agp_backend_initialize() failed. agpgart-serverworks: probe of 0000:00:00.0 failed with error -22 agpgart: unable to determine aperture size. agpgart: agp_backend_initialize() failed. agpgart-serverworks: probe of 0000:00:00.1 failed with error -22 agpgart: ServerWorks CNB20HE is unsupported due to lack of documentation. agpgart: ServerWorks CNB20HE is unsupported due to lack of documentation. Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled erial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a NS16550A ? ??? ? ?? ??? ?? ?? ? ? ?? ? ??? ? ? ?? ? ? ? ?? ?? ? ?? ? ? ? ? ??? ? ? ? ?? ??? ? ??? ??? ? ??? ? ? ? ? ? ??????? ????? ???? ? ?????? ? ??? ? ??? ? ????? ??? ?? ???? ????? ?? ????? ?? ??? ????? ?? ??? ? ?? ???? ????? ?? ???? ? ? ??? ? ? ?? ?? ???? ? ? ???? ? ??? ??? ?? ????? ?? ? ?? ? ? ? ?? ? ??? ? ?? ? ? ?? ??? ? ? ? ?? ?? ?????????? ?? ??? ? ? ? ?? ? ? ?? ? ?? ?? ? ???? ????? ?????? ? ? ? ??? ??? ? ? ???? ? ??? ?? ? ??????? ??? ?? ??? ??? ???? ? ? ?? ?? ??? ? ???? ?? ?? ?? ? ?? ??? ? ?????? ??? ? ??? ? ? ?? ?? ?????????? ? ? ?? ?? ???? ? ? ? ??? ? ? ???? ?? ?? ???? ?? ???? ? ? ? ? ?? ??? ?? ? ? ? ????? ?? ? ? ? ? ? ?? ? ? ?? ?? ?????????? ? ? ?? ?? ???? ? ? ? ??? ?? ?? ??? ? ?? ??? ? ??? ??? ?? ????? ?? ? ? ?? ?? ? ? ?? ?? ?????????? ? ? ?? ?? ???? ? ????? ?? ? ?? ??? ? ??? ?? ???? ? ??????? ??? ? ? ???? ?? ?? ?? ? ???? ??? ? ? ?? ?? ?? ? ??? ? ???? ? ??? ? ? ?? ?? ?????????? ? ? ?? ?? ???? ? ? ?? ?????? ? ???? ? ? ??? ? ??? ??? ??? ? ? ??? ?? ? ???? ?? ??? ? ??? ? ? ?? ??? ? ? ??? ???? ??? ?? ? ? ? ?? ? ? ??? ? ??? ? ?? ?? ?? ?? ? ??? ? ?? ??? ?? ??? ?? ? ? ?? ?? ? ? ?? ????? ? ?? ? ? ? ?? ??????? ? ? ?? ?? ?????????? ? ? ?? ?? ???? ? ? ?? ?????? ? ???? ? ? ??? ? ??? ??? ??? ? ? ??? ?? ? ???? ?? ??? ? ??? ? ? ?? ??? ?? ? ??? ???? ??? ?? ? ? ? ?? ? ? ??? ? ? ? ? ?? ?? ?? ?? ? ??? ? ?? ??? ?? ??? ?? ? ? ?? ?? ? ? ?? ????? ? ?? ? ? ? ?? ??????? ? ?? ?? ? ? ? ??? ??? ?? ??????? ?? ? ? ? ???? ??? ??? ? ? ?? ???? ? ? ? ? ? ???? ? ??? ? ? ? ??? ? ??? ???? ??? ??? ? ? ?? ?? ???? ??? ? ? ?? ??? ? ? ? ?? ?? ? ??? ?? ??? ? ?? ? ? ??? ?? ? ?? ?? ? ? ? ?? ? ? ?? ??? ???? ? ????? ? ?? ? ? ?? ?? ???? ? ? ? ? ? ? ???? ????? ?? ??? ??? ? ????? ? ?? ???? ??? ? ? ??? ?? ?? ??? ? ???? ? ??? ? ? ???? ???? ? ?? ?? ???? ? ? ??? ?? ? ??? ? ???? ? ????? ? ?? ???? ??? ? ? ??? ?? ?? ??? ? ???? ? ??? ? ? ???? ???? ? ?? ?? ???? ? ? ????? ? ? ??? ?? ??? ? ? ?? ?? ???? ? ?? ??? ????? ? ??? ? ?? ?? ????? ???? ??? ?? ???? ? ???? ??? ? ? ? ? ? ?? ? ? ? ?????? ?? ? ?? ? ? ? ???? ??? ? ? ?? ?? ???? ? ??? ? ?? ? ?? ? ???? ??? ???? ? ???? ?? ?? ???? ?? ???? ? ? ???????? ?? ? ? ?? ???? ?? ?? ??? ? ???? ? ??? ? ?? ?? ??? ?? ? ?? ??? ? ? ?? ? ? ??? ? ?? ????? ? ? ?? ? ?? ? ? ? ???? ? ?? ? ? ?? ? ? ? ? ? ? ? ?? ?? ? ? ?? ? ? ??? ? ?? ????? ? ? ?? ? ? ?????? ?? ???? ? ?? ? ? ?? ? ? ? ? ? ? ? ?? ?? ? ? ?? ? ????? ??? ??? ? ? ??? ???? ???????????? ??? ? ??? ? ??? ??? ??? ? ? ??????????????? ?? ???? ? ? ? ?? ?? ????? ??? ??? ??? ? ???? ? ? ???? ? ? ?? ?? ???? ? ? ? ? ? ? ?? ?? ? ??????? ? ???? ??? ? ???? ???? ??? ? ? ?? ? ????? ?? ???? ?? ?? ???? ?? ???? ?? ?? ? ???? ? ? ?? ?? ??????? ? ? ???? ? ?? ?? ????? ?? ? ? ? ? ?? ? ???? ? ?? ?? ??? ? ??? ?? ?? ??? ? ???? ? ??? ? ??? ? ? ? ??? ?? ??? ??? ? ??? ?? ??? ?? ??? ? ? ? ??? ??? ?? ? ???? ? ?? ??? ?? ?????? ? ????? ??????? ? ???? ? ??? ? ??? ? ???? ?? ?? ??? ? ???? ? ??? ? ? ? ???? ?????? ???? ?? ?? ?????? ?? ??? ? ? ?? ? ??? ? ??? ? ???? ?? ?? ??? ? ???? ? ??? ? ? ? ???? ?????? ???? ?? ?? ?????? ?? ??? ? ? ?? ?? ? ?????? ?? ?? ??? ? ? ? ??? ? ??????? ? ?? ????? ? ??? ????? ?? ??????? ??? ??? ? ? ?? ? ??? ? ? ? ? ????? ? ???? ? ??? ? ? ???? ???????? ? ? ??? ?? ? ? ? ?????? ?????? ???????? ?? ??? ? ??? ?? ? ??? ??? ? ?? ? ?? ??? ? ??? ? ?? ???? ? ? ? ?? ? ? ?? ? ?? ? ? ????? ? ? ?? ? ? ? ??? ?? ??? ? ??? ?? ??? ?? ??? ? ???? ? ???????? ????? ? ???? ? ??????? ??? ?? ????? ? ? ??? ? ? ??? ??? ??? ??? ? ? ??? ??? ?? ?? ??? ? ???? ??????? ??? ? ? ?????? ? ? ???? ? ?? ?????? ???????? ?? ? ? ? ???? ? ??? ??? ? ??? ? ???? ?? ?? ???? ? ? ???? ???? ?? ? ??? ?? ??? ? ?? ???? ? ?????? ? ? ???? ? ?? ??? ?? ? ???? ??? ?? ??? ? ?? ? ??? ? ? ? ? ? ? ? ? ? ?????? ? Mounting proc filesystem Mounting sysfs filesystem Creating /dev Creating initial device nodes Loading scsi_mod.ko module SCSI subsystem initialized Loading sd_mod.ko module Loading aacraid.ko module Adaptec aacraid driver 1.1-5[24702] ACPI: PCI Interrupt 0000:01:08.1[A] -> GSI 18 (level, low) -> IRQ 177 BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<ca86dcf6>] CPU: 0 EIP is at aac_rx_check_health+0x16/0x1d4 [aacraid] EFLAGS: 00000282 Not tainted (2.6.18-267.el5PAE #1) EAX: ca814000 EBX: ffffffff ECX: a6b518c1 EDX: 00000080 ESI: c9940020 EDI: c9924ae0 EBP: 018e7ed8 DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f07ff>] __down_trylock+0x3b/0x44 [<ca86ca56>] aac_fib_send+0x228/0x2ee [aacraid] [<ca86930d>] aac_get_adapter_info+0x50/0x58a [aacraid] [<ca866e47>] aac_probe_one+0x211/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= [-- MARK -- Mon Jun 13 02:45:00 2011] BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<c20f212f>] CPU: 0 EIP is at delay_tsc+0xb/0x13 EFLAGS: 00000283 Not tainted (2.6.18-267.el5PAE #1) EAX: 30c115c3 EBX: 00001b47 ECX: 30c103f9 EDX: 000000aa ESI: c9940020 EDI: c9924ae0 EBP: 00f7ab3c DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f2160>] __delay+0x6/0x7 [<ca86ca8b>] aac_fib_send+0x25d/0x2ee [aacraid] [<ca86930d>] aac_get_adapter_info+0x50/0x58a [aacraid] [<ca866e47>] aac_probe_one+0x211/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<c20f212f>] CPU: 0 EIP is at delay_tsc+0xb/0x13 EFLAGS: 00000297 Not tainted (2.6.18-267.el5PAE #1) EAX: ba77b97b EBX: 00001b47 ECX: ba77a4c5 EDX: 000000bd ESI: c9940020 EDI: c9924ae0 EBP: 0060d821 DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f2160>] __delay+0x6/0x7 [<ca86ca8b>] aac_fib_send+0x25d/0x2ee [aacraid] [<ca86930d>] aac_get_adapter_info+0x50/0x58a [aacraid] [<ca866e47>] aac_probe_one+0x211/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= aacraid: aac_fib_send: first asynchronous command timed out. Usually a result of a PCI interrupt routing problem; update mother board BIOS or consider utilizing one of the SAFE mode kernel options (acpi, apic etc) BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<c20f212f>] CPU: 0 EIP is at delay_tsc+0xb/0x13 EFLAGS: 00000297 Not tainted (2.6.18-267.el5PAE #1) EAX: cbe49f83 EBX: 00001b47 ECX: cbe49bab EDX: 000000dd ESI: c9940020 EDI: c9924ae0 EBP: 018e811c DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f2160>] __delay+0x6/0x7 [<ca86ca8b>] aac_fib_send+0x25d/0x2ee [aacraid] [<ca86b712>] aac_send_shutdown+0x45/0x6b [aacraid] [<ca866035>] __aac_shutdown+0x1e/0x4d [aacraid] [<ca866f75>] aac_probe_one+0x33f/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<c20f212f>] CPU: 0 EIP is at delay_tsc+0xb/0x13 EFLAGS: 00000293 Not tainted (2.6.18-267.el5PAE #1) EAX: 55f07e4b EBX: 00001b47 ECX: 55f07237 EDX: 000000f1 ESI: c9940020 EDI: c9924ae0 EBP: 00f7b262 DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f2160>] __delay+0x6/0x7 [<ca86ca8b>] aac_fib_send+0x25d/0x2ee [aacraid] [<ca86b712>] aac_send_shutdown+0x45/0x6b [aacraid] [<ca866035>] __aac_shutdown+0x1e/0x4d [aacraid] [<ca866f75>] aac_probe_one+0x33f/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= [-- MARK -- Mon Jun 13 02:50:00 2011] BUG: soft lockup - CPU#0 stuck for 60s! [insmod:353] Pid: 353, comm: insmod EIP: 0060:[<c20f212f>] CPU: 0 EIP is at delay_tsc+0xb/0x13 EFLAGS: 00000287 Not tainted (2.6.18-267.el5PAE #1) EAX: e011abb9 EBX: 00001b47 ECX: e011a869 EDX: 00000104 ESI: c9940020 EDI: c9924ae0 EBP: 0060e1fa DS: 007b ES: 007b CR0: 8005003b CR2: 08652110 CR3: 02343380 CR4: 000006f0 [<c20f2160>] __delay+0x6/0x7 [<ca86ca8b>] aac_fib_send+0x25d/0x2ee [aacraid] [<ca86b712>] aac_send_shutdown+0x45/0x6b [aacraid] [<ca866035>] __aac_shutdown+0x1e/0x4d [aacraid] [<ca866f75>] aac_probe_one+0x33f/0x3c6 [aacraid] [<c21609b4>] __driver_attach+0x0/0x6b [<c20fbd09>] pci_device_probe+0x36/0x57 [<c2160907>] driver_probe_device+0x54/0xa4 [<c21609f8>] __driver_attach+0x44/0x6b [<c21602d5>] bus_for_each_dev+0x37/0x59 [<c2160840>] driver_attach+0x11/0x13 [<c21609b4>] __driver_attach+0x0/0x6b [<c215ffa6>] bus_add_driver+0x64/0xfd [<c20fbf67>] __pci_register_driver+0x42/0x8c [<ca802023>] aac_init+0x23/0x58 [aacraid] [<c203fe06>] sys_init_module+0x1af3/0x1cb8 [<c2072bc1>] __kmalloc+0x0/0x72 [<c206866c>] sys_mmap_pgoff+0x6d/0x81 [<c2004f4b>] syscall_call+0x7/0xb ======================= aac_fib_free, XferState != 0, fibptr = 0xc9940000, XferState = 0x810ad ACPI: PCI interrupt for device 0000:01:08.1 disabled aacraid: probe of 0000:01:08.1 failed with error -110 Loading jbd.ko module Loading ext3.ko module Loading dm-mod.ko module device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.11.6-ioctl (2011-02-18) initialised: dm-devel Loading dm-log.ko module Loading dm-mirror.ko module Loading dm-zero.ko module Loading dm-snapshot.ko module Loading e1000.ko module Intel(R) PRO/1000 Network Driver - version 7.3.21-k4-3-NAPI Copyright (c) 1999-2006 Intel Corporation. ACPI: PCI Interrupt 0000:01:02.0[A] -> GSI 17 (level, low) -> IRQ 185 e1000: 0000:01:02.0: e1000_probe: (PCI:66MHz:64-bit) 00:06:5b:0f:7c:d9 e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection ACPI: PCI Interrupt 0000:01:04.0[A] -> GSI 16 (level, low) -> IRQ 193 e1000: 0000:01:04.0: e1000_probe: (PCI:66MHz:64-bit) 00:06:5b:0f:7c:da e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection Waiting for required block device discovery Waiting for sda...BUG: unable to handle kernel paging request at virtual address 756e6766 printing eip: ca86d2e9 *pde = cccccccc Oops: 0000 [#1] SMP last sysfs file: Modules linked in: e1000 dm_snapshot dm_zero dm_mirror dm_log dm_mod ext3 jbd aacraid sd_mod scsi_mod CPU: 0 EIP: 0060:[<ca86d2e9>] Not tainted VLI EFLAGS: 00010246 (2.6.18-267.el5PAE #1) EIP is at aac_command_thread+0x3ec/0x5f1 [aacraid] eax: 756e672e ebx: c9924ae0 ecx: 00000286 edx: c9924ae0 esi: c9e1eda8 edi: 0002aa4e ebp: 00000000 esp: c9e6cf54 ds: 007b es: 007b ss: 0068 Process aacraid (pid: 358, ti=c9e6c000 task=c9f2caa0 task.ti=c9e6c000) Stack: c9e6cfc8 c221fe7a e086d801 0000005d 00000000 c9924ae0 00000000 0002aa4e 0002aa4e e086f382 0000005d 00001b81 00000000 c9f2cbac c25ac880 c2344ac0 00000003 c9e1edac 00000001 00000000 c9f2caa0 c201fc7b c9938430 c9938430 Call Trace: [<c221fe7a>] schedule+0x9ce/0xa57 [<c201fc7b>] default_wake_function+0x0/0xc [<ca86cefd>] aac_command_thread+0x0/0x5f1 [aacraid] [<c203706a>] kthread+0xc0/0xee [<c2036faa>] kthread+0x0/0xee [<c2005c87>] kernel_thread_helper+0x7/0x10 ======================= Code: 80 f7 83 c6 04 3b 5c 24 10 72 e5 8b 44 24 28 e8 8c 4f 80 f7 8b 44 24 2c e8 83 4f 80 f7 89 e8 e8 7c 4f 80 f7 8b 54 24 14 8b 42 34 <8b> 40 38 e8 a7 42 9b f7 8b 4c 24 14 89 c7 8b 51 34 8d 42 40 39 EIP: [<ca86d2e9>] aac_command_thread+0x3ec/0x5f1 [aacraid] SS:ESP 0068:c9e6cf54 <0>Kernel panic - not syncing: Fatal exception [-- MARK -- Mon Jun 13 02:55:00 2011] [-- MARK -- Mon Jun 13 03:00:00 2011] [-- MARK -- Mon Jun 13 03:05:00 2011] [-- MARK -- Mon Jun 13 03:10:00 2011] [-- MARK -- Mon Jun 13 03:15:00 2011] [-- MARK -- Mon Jun 13 03:20:00 2011] [-- MARK -- Mon Jun 13 03:25:00 2011] [-- MARK -- Mon Jun 13 03:30:00 2011] [-- MARK -- Mon Jun 13 03:35:00 2011] [-- MARK -- Mon Jun 13 03:40:00 2011] [-- MARK -- Mon Jun 13 03:45:00 2011] [-- MARK -- Mon Jun 13 03:50:00 2011] [-- MARK -- Mon Jun 13 03:55:00 2011] [-- MARK -- Mon Jun 13 04:00:00 2011] [-- MARK -- Mon Jun 13 04:05:00 2011] [-- MARK -- Mon Jun 13 04:10:00 2011] Version-Release number of selected component (if applicable): 2.6.18-267.el5 How reproducible: not sure Steps to Reproduce: 1. install dom0 on dell-pe1650-02.rhts.eng.bos.redhat.com 2. configure kdump on dell-pe1650-02.rhts.eng.bos.redhat.com using network target: net root.xxx.xxx link_delay 60 core_collector makedumpfile -E -d 1 3. trigger crash with sysrq-c Actual results: capture kernel will panic Expected results: Additional info: