Bug 254641 - (mm/filemap.c:470) PANIC: "<4>kernel BUG at mm/filemap.c:470!"
PANIC: "<4>kernel BUG at mm/filemap.c:470!"
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel (Show other bugs)
4.4
ia64 Linux
medium Severity high
: ---
: ---
Assigned To: Luming Yu
Martin Jenner
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-08-25 23:54 EDT by Eduardo Alvarenga
Modified: 2013-08-05 21:43 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-10-23 10:33:19 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Eduardo Alvarenga 2007-08-25 23:54:44 EDT
Description of problem:


Version-Release number of selected component (if applicable):

======================
CRASH CORE FILE REPORT
======================

GENERATED ON:
    Thu Aug 25 00:54:27 BRT 2007

==================
SYSTEM INFORMATION
==================

      KERNEL: /usr/lib/debug/lib/modules/2.6.9-42.0.8.EL/vmlinux
    DUMPFILE: /var/crash/127.0.0.1-2007-08-25-00:45/vmcore
        CPUS: 4
        DATE: Thu Aug 25 00:45:20 2007
      UPTIME: 11 days, 01:33:09
LOAD AVERAGE: 3.45, 1.00, 1.32
       TASKS: 454
    NODENAME: <hidden>
     RELEASE: 2.6.9-42.0.8.EL
     VERSION: #1 SMP Tue Jan 23 12:34:40 EST 2007
     MACHINE: ia64  (1599 Mhz)
      MEMORY: 10 GB
       PANIC: "<4>kernel BUG at mm/filemap.c:470!"
    SOFTWARE: Oracle 10g RAC DB 10.1.0.2 + ASMLib 2.0.2-1
              HP Integrity Essentials Foundations Pack for Linux version 3.63
            


===================
MACHINE INFORMATION
===================

              MACHINE TYPE: ia64
               MEMORY SIZE: 10 GB
                      CPUS: 4
           PROCESSOR SPEED: 1599 Mhz
                        HZ: 1024
                 PAGE SIZE: 16384
             L1 CACHE SIZE: 128
         KERNEL STACK SIZE: 32768
      KERNEL CACHED REGION: e000000000000000
    KERNEL UNCACHED REGION: c000000000000000
     KERNEL VMALLOC REGION: a000000000000000
    USER DATA/STACK REGION: 8000000000000000
    USER DATA/STACK REGION: 6000000000000000
          USER TEXT REGION: 4000000000000000
 USER SHARED MEMORY REGION: 2000000000000000
USER IA32 EMULATION REGION: 0000000000000000


===============
LOG BUFFER DUMP
===============

<4>kernel BUG at mm/filemap.c:470!
kipmi0[7169]: bugcheck! 0 [1]
Modules linked in: sg md5 ipv6 oracleasm(U) autofs4 ide_dump cciss_dump
scsi_dump diskdump zlib_deflate ipmi_devintf ipmi_si ipmi_msghandler dev_acpi(U)
vfat
 fat dm_mirror button joydev ohci_hcd ehci_hcd e1000 bond2(U) bond1(U) bond0(U)
ext3 jbd dm_mod qla2400(U) qla2300(U) qla2xxx(U) cciss qla2xxx_conf(U) sd_mod
 scsi_mod

Pid: 7169, CPU 0, comm:               kipmi0
psr : 0000101008022018 ifs : 8000000000000207 ip  : [<a0000001000d2c50>]    Not
tainted
ip is at unlock_page+0x90/0xe0
unat: 0000000000000000 pfs : 0000000000000207 rsc : 0000000000000003
rnat: 0000000000000000 bsps: a000000200abc000 pr  : 000a06a60002a595
ldrs: 0000000000000000 ccv : 0000000000000004 fpsr: 0009804c8a70033f
csd : 0000000000000000 ssd : 0000000000000000
b0  : a0000001000d2c50 b6  : a0000001000724e0 b7  : a00000010045c720
f6  : 1003e0000000000001200 f7  : 1003e8080808080808081
f8  : 1003e00000000000023dc f9  : 1003e000000000e580000
f10 : 1003e00000000356f424c f11 : 1003e44b831eee7285baf
r1  : a0000001009bb9e0 r2  : e000000001004ab8 r3  : e000000001010000
r8  : 0000000000000020 r9  : 0000000000000000 r10 : a0000001007d7050
r11 : a0000001007d7050 r12 : e00000027ec37c50 r13 : e00000027ec30000
r14 : 0000000000004000 r15 : a00000010074fbc0 r16 : a00000010074fbc8
r17 : e000000101e6fde8 r18 : 0000000000000001 r19 : 0000000000000004
r20 : 0000000000000004 r21 : 0000000000000004 r22 : 0000000000000001
r23 : 0000000000000000 r24 : 0000000000000000 r25 : 0000000000000004
r26 : e00000027ec30dd0 r27 : 0000000000000074 r28 : 0000000000000074
r29 : 0000000000000065 r30 : e000000101e68050 r31 : 00000000356f424c

Call Trace:
 [<a000000100016da0>] show_stack+0x80/0xa0
                                sp=e00000027ec377c0 bsp=e00000027ec31238
 [<a0000001000176b0>] show_regs+0x890/0x8c0
                                sp=e00000027ec37990 bsp=e00000027ec311e8
 [<a00000010003e8f0>] die+0x150/0x240
                                sp=e00000027ec379b0 bsp=e00000027ec311a8
 [<a00000010003ea20>] die_if_kernel+0x40/0x60
                                sp=e00000027ec379b0 bsp=e00000027ec31178
 [<a00000010003ebc0>] ia64_bad_break+0x180/0x600
                                sp=e00000027ec379b0 bsp=e00000027ec31150
 [<a00000010000f600>] ia64_leave_kernel+0x0/0x260
                                sp=e00000027ec37a80 bsp=e00000027ec31150
 [<a0000001000d2c50>] unlock_page+0x90/0xe0
                                sp=e00000027ec37c50 bsp=e00000027ec31118
 [<a00000010017a8b0>] mpage_end_io_read+0x110/0x220
                                sp=e00000027ec37c50 bsp=e00000027ec310e0
 [<a000000100133110>] bio_endio+0x110/0x1c0
                                sp=e00000027ec37c50 bsp=e00000027ec310a8
 [<a0000002000c5020>] do_cciss_intr+0x1100/0x4760 [cciss]
                                sp=e00000027ec37c50 bsp=e00000027ec30fb8
 [<a0000001000130f0>] handle_IRQ_event+0x90/0x120
                                sp=e00000027ec37c50 bsp=e00000027ec30f78
 [<a000000100013c30>] do_IRQ+0x2d0/0x560
                                sp=e00000027ec37c50 bsp=e00000027ec30f10
 [<a000000100015d10>] ia64_handle_irq+0xf0/0x1e0
                                sp=e00000027ec37c50 bsp=e00000027ec30ec8
 [<a00000010000f600>] ia64_leave_kernel+0x0/0x260
                                sp=e00000027ec37c50 bsp=e00000027ec30ec8
 [<a0000001000aea00>] kthread_should_stop+0x0/0x40
                                sp=e00000027ec37e20 bsp=e00000027ec30eb8
 [<a0000002008be6e0>] ipmi_thread+0x60/0x1a0 [ipmi_si]
                                sp=e00000027ec37e20 bsp=e00000027ec30e70
 [<a0000001000aec20>] kthread+0x1e0/0x240
                                sp=e00000027ec37e20 bsp=e00000027ec30e38
 [<a000000100018c70>] kernel_thread_helper+0x30/0x60
                                sp=e00000027ec37e30 bsp=e00000027ec30e10
 [<a000000100008c60>] start_kernel_thread+0x20/0x40
                                sp=e00000027ec37e30 bsp=e00000027ec30e10
fffa564, ip=0x200000000266a3e0
ons(31111): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31111): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
ons(31111): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31111): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
ons(31112): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31112): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
ons(31112): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31112): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
ons(31006): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31006): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
ons(31006): unaligned access to 0x60000fffffffa55c, ip=0x200000000266a3c0
ons(31006): unaligned access to 0x60000fffffffa564, ip=0x200000000266a3e0
[...]

====================
CURRENT SYSTEM TASKS
====================

   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
      0      0   0  a0000001006e0000  RU   0.0       0      0  [swapper]
      0      1   1  e00000027ffd8000  RU   0.0       0      0  [swapper]
      0      1   2  e000000001470000  RU   0.0       0      0  [swapper]
      0      1   3  e0000040ffad8000  RU   0.0       0      0  [swapper]
      1      0   0  e000000003b50000  IN   0.0    3440   1344  init
      2      1   0  e000000001428000  IN   0.0       0      0  [migration/0]
      3      1   0  e000000001430000  IN   0.0       0      0  [ksoftirqd/0]
      4      1   1  e000000001438000  IN   0.0       0      0  [migration/1]
      5      1   1  e00000027ffd0000  IN   0.0       0      0  [ksoftirqd/1]
      6      1   2  e00000003fe00000  IN   0.0       0      0  [migration/2]
      7      1   2  e000000001468000  IN   0.0       0      0  [ksoftirqd/2]
      8      1   3  e000000001478000  IN   0.0       0      0  [migration/3]
      9      1   3  e0000040ffad0000  IN   0.0       0      0  [ksoftirqd/3]
     10      1   0  e0000040fed68000  IN   0.0       0      0  [events/0]
     11      1   1  e00000027fec8000  IN   0.0       0      0  [events/1]
     12      1   2  e0000040fed70000  IN   0.0       0      0  [events/2]
     13      1   3  e0000040fed78000  IN   0.0       0      0  [events/3]
     14     10   3  e00000027fed0000  IN   0.0       0      0  [khelper]
     15     10   0  e0000001001a0000  IN   0.0       0      0  [kacpid]
     46     10   0  e0000001001a8000  IN   0.0       0      0  [kblockd/0]
     47     10   1  e000000006370000  IN   0.0       0      0  [kblockd/1]
     48     10   2  e000000006378000  IN   0.0       0      0  [kblockd/2]
     49     10   3  e000000006380000  IN   0.0       0      0  [kblockd/3]
     50      1   2  e0000000063d8000  IN   0.0       0      0  [khubd]
>    69      1   1  e00000027fc80000  RU   0.0       0      0  [kswapd0]
     70     10   0  e00000027fcd8000  IN   0.0       0      0  [aio/0]
     71     10   1  e00000027fce0000  IN   0.0       0      0  [aio/1]
     72     10   2  e00000027fce8000  IN   0.0       0      0  [aio/2]
     73     10   3  e00000027fcf0000  IN   0.0       0      0  [aio/3]
    216      1   0  e000000100260000  IN   0.0       0      0  [kseriod]
    305  32743   1  e000000186340000  IN   0.1  199888  11408  ocssd.bin
    308  30265   0  e000000191e08000  IN   0.1  180496  10768  evmd.bin
    320  30265   1  e0000001920e8000  IN   0.1  180496  10768  evmd.bin
    321  30265   2  e0000001ae648000  RU   0.1  180496  10768  evmd.bin
    322  30265   1  e0000001adfd8000  IN   0.1  180496  10768  evmd.bin
    323  30265   3  e000000147e40000  IN   0.1  180496  10768  evmd.bin
    324  30265   0  e0000001ae6e8000  IN   0.1  180496  10768  evmd.bin
    325  30265   1  e000000192a58000  IN   0.1  180496  10768  evmd.bin
    326  30265   3  e0000001ae8b8000  IN   0.1  180496  10768  evmd.bin
    327  30265   3  e0000001adf68000  IN   0.1  180496  10768  evmd.bin
    328  30265   3  e0000001adf90000  IN   0.1  180496  10768  evmd.bin
    329    323   2  e000000192af8000  IN   0.1   58976   5888  evmlogger.bin
    346  32743   0  e0000001adfe0000  IN   0.1  199888  11408  ocssd.bin
    377      1   3  e0000000064f8000  IN   0.0       0      0  [scsi_eh_0]
    378      1   3  e00000003da18000  IN   0.0       0      0  [qla2300_0_dpc]
    380      1   3  e00000003d940000  IN   0.0       0      0  [scsi_eh_1]
    381      1   3  e000000006548000  IN   0.0       0      0  [qla2300_1_dpc]
    383      1   3  e0000001007c0000  IN   0.0       0      0  [scsi_eh_2]
    384      1   3  e00000003d980000  IN   0.0       0      0  [qla2300_2_dpc]
    447  32743   1  e000000193f98000  RU   0.1  199888  11408  ocssd.bin
    448  32743   3  e0000001a0b28000  IN   0.1  199888  11408  ocssd.bin
    449  32743   3  e0000001adf78000  IN   0.1  199888  11408  ocssd.bin
    450  32743   2  e0000001924e8000  IN   0.1  199888  11408  ocssd.bin
    451  32743   0  e000004093dd8000  IN   0.1  199888  11408  ocssd.bin
    452  32743   3  e0000001ae168000  IN   0.1  199888  11408  ocssd.bin
    453  32743   3  e0000001adfc0000  IN   0.1  199888  11408  ocssd.bin
    454  32743   2  e0000001a79e0000  IN   0.1  199888  11408  ocssd.bin
    455  32743   1  e0000001ae398000  IN   0.1  199888  11408  ocssd.bin
    456  32743   3  e0000001ae898000  IN   0.1  199888  11408  ocssd.bin
    472  30265   3  e0000001ae010000  IN   0.1  180496  10768  evmd.bin
    664      1   3  e00000027fd90000  IN   0.0       0      0  [scsi_eh_3]
    665      1   0  e0000040fe350000  IN   0.0       0      0  [qla2300_3_dpc]
    690      1   0  e0000040ff088000  IN   0.0       0      0  [kjournald]
    858      1   0  e0000001ae750000  IN   0.0   39696    496  ons
    859    858   0  e0000001ae2e8000  IN   0.0   53296   3168  ons
    862    859   2  e0000001ae368000  IN   0.0   53296   3168  ons
    863    862   3  e0000001ae2a0000  IN   0.0   53296   3168  ons
    864    862   3  e0000001ae780000  IN   0.0   53296   3168  ons
    865    862   3  e0000001ae880000  IN   0.0   53296   3168  ons
    866    862   0  e0000001ae850000  IN   0.0   53296   3168  ons
    867    862   0  e0000001ae800000  IN   0.0   53296   3168  ons
    868    862   2  e0000001ae838000  IN   0.0   53296   3168  ons
    882  14858   2  e000000230830000  IN   0.1   14512   7248  sshd
    917    882   2  e0000001c4758000  IN   0.1   14512   5840  sshd
    918    917   1  e00000023e4f8000  IN   0.0   55296   3712  bash
   1114  14858   2  e00000019d8b8000  IN   0.1   14512   7392  sshd
   1116   1114   1  e000000028600000  IN   0.1   14512   5952  sshd
   1130   1116   1  e00000014a538000  IN   0.0   55504   4336  bash
   3197     13   3  e0000000110c8000  IN   0.0       0      0  [pdflush]
   3397      1   3  e00000027efe0000  IN   0.0    3152   1008  udevd
   3886      1   1  e00000010d9b8000  IN   0.1  493872   8688  oracle
   3888      1   0  e00000013b250000  IN   0.1  494016   6896  oracle
   3890      1   3  e000004091710000  IN   0.1  496064   8272  oracle
   3892      1   3  e00000015f540000  IN   0.1  507072   7248  oracle
   3894      1   3  e0000001adae0000  IN   0.1  507088   7056  oracle
   3896      1   3  e000000146bb8000  IN   0.1  492592   6304  oracle
   3898      1   0  e0000001449d0000  IN   0.1  496720   7232  oracle
   3900      1   3  e000000137268000  IN   0.1  493760   7648  oracle
   3902      1   0  e00000019c190000  IN   0.1  496960   8528  oracle
   3904      1   0  e00000408b5f8000  IN   0.1  494016   7904  oracle
   3906      1   3  e0000001672f0000  IN   0.1  493872   7520  oracle
   3960      1   3  e00000408a840000  IN   0.1  494016   9376  oracle
   4009      1   3  e000000144a60000  IN   0.1   91536   5952  racgimon
   4011      1   0  e000000173778000  IN   0.1   91536   5952  racgimon
   4114      1   3  e000000144510000  IN   0.1   91536   5952  racgimon
   4115      1   1  e00000010d918000  IN   0.1   91536   5952  racgimon
   4178      1   2  e000000037140000  IN   7.6 5009104 791552  oracle
   4322      1   3  e0000040f7fb8000  IN   0.1   91536   5952  racgimon
   6156      1   2  e0000040c3498000  IN  15.1 4707360 1578032  oracle
   6684  14858   2  e000000167c60000  IN   0.1   14512   7248  sshd
   6707   6684   2  e000004042fd8000  IN   0.1   14512   5904  sshd
   6708   6707   3  e0000002482e0000  IN   0.0   55296   3712  bash
   6772     10   0  e000000101680000  IN   0.0       0      0  [kauditd]
   6816      1   0  e000000196910000  IN   3.5 4687712 369424  oracle
   6855     10   0  e00000027f198000  IN   0.0       0      0  [kmirrord]
   7015      1   0  e000000102fe0000  IN   0.0       0      0  [kjournald]
   7016      1   0  e000000102f08000  IN   0.0       0      0  [kjournald]
   7017      1   0  e0000040fe3c8000  IN   0.0       0      0  [kjournald]
   7018      1   0  e000000103e58000  IN   0.0       0      0  [kjournald]
   7019      1   0  e0000001036e8000  IN   0.0       0      0  [kjournald]
   7020      1   0  e000000103970000  IN   0.0       0      0  [kjournald]
   7021      1   0  e0000040ff168000  IN   0.0       0      0  [kjournald]
   7022      1   3  e000000103398000  IN   0.0       0      0  [kjournald]
   7023      1   0  e000000101d38000  IN   0.0       0      0  [kjournald]
   7059      1   2  e00000406b738000  IN   2.9 4687696 299088  oracle
   7163      1   2  e000000162340000  RU   3.8 4687712 391904  oracle
>  7169     10   0  e00000027ec30000  RU   0.0       0      0  [kipmi0]
   7196      1   2  e00000001d6b0000  IN   0.2 4683024  24016  oracle
   7499      1   0  e00000027f1b0000  IN   0.0    6448   1280  hpseld
   8015      1   2  e000000119430000  IN   0.0   49344   2560  arelay
   8016      1   1  e00000404d178000  IN   0.0   49344   2560  arelay
   8017      1   1  e0000002319c0000  IN   0.0   49344   2560  arelay
   8018      1   1  e00000001cf28000  IN   0.0   49344   2560  arelay
   8022      1   2  e000000219a00000  IN   0.0   29936   3808  aengine
   8023      1   1  e00000406ca30000  IN   0.0   29936   3808  aengine
   8127      1   3  e0000040fb700000  IN   0.0    4128   1536  syslogd
   8131      1   0  e000000101e68000  RU   0.0    3936   1120  klogd
   8227      1   0  e0000000168c0000  IN   0.0   29936   3808  aengine
   8229      1   0  e0000001e0cf8000  RU   0.0   49344   2560  arelay
   8359      1   1  e0000001ae488000  IN   0.2  554496  21456  crsd.bin
   9291      1   2  e00000022f730000  IN  18.9 5025824 1965664  oracle
  10085  14858   2  e000000015ad8000  IN   0.0   14512   4288  sshd
  10148  10085   2  e0000040cb9d8000  IN   0.0   14512   3504  sshd
  10159  10148   2  e000000017e48000  IN   0.0   55296   3232  bash
  11249      1   0  e000000022660000  IN   0.2  554496  21456  crsd.bin
  11449      1   3  e000004075500000  IN   0.0    3664    784  start.sh
  11488      1   2  e000000170198000  IN   7.6 5009152 797184  oracle
  11786      1   1  e0000002657e8000  IN   0.2  554496  21456  crsd.bin
  11787      1   3  e000000036cd0000  IN   0.2  554496  21456  crsd.bin
  11788      1   3  e0000040f5390000  IN   0.2  554496  21456  crsd.bin
  11789      1   1  e000000249950000  IN   0.2  554496  21456  crsd.bin
  11790      1   1  e000000151998000  IN   0.2  554496  21456  crsd.bin
  11791      1   1  e000000265be8000  IN   0.2  554496  21456  crsd.bin
  11792      1   3  e0000001b21d0000  IN   0.2  554496  21456  crsd.bin
  11793      1   3  e0000040788e0000  IN   0.2  554496  21456  crsd.bin
  11796      1   1  e00000019c4b0000  IN   0.2  554496  21456  crsd.bin
  11797      1   1  e00000019bf18000  IN   0.2  554496  21456  crsd.bin
  11798      1   0  e000000149f48000  IN   0.2  554496  21456  crsd.bin
  11799      1   3  e000000217d00000  IN   0.2  554496  21456  crsd.bin
  11800      1   0  e000000238578000  IN   0.2  554496  21456  crsd.bin
  11801      1   0  e0000040c1880000  IN   0.2  554496  21456  crsd.bin
  11802      1   0  e000000174f58000  IN   0.2  554496  21456  crsd.bin
  11803      1   0  e000000034b60000  IN   0.2  554496  21456  crsd.bin
  11804      1   0  e0000040e7628000  IN   0.2  554496  21456  crsd.bin
  11805      1   0  e000000197890000  IN   0.2  554496  21456  crsd.bin
  11806      1   0  e0000040bb308000  IN   0.2  554496  21456  crsd.bin
  11807      1   0  e00000023ea38000  IN   0.2  554496  21456  crsd.bin
  11808      1   0  e000000030800000  IN   0.2  554496  21456  crsd.bin
  11809      1   3  e000000101140000  IN   0.2  554496  21456  crsd.bin
  11868      1   1  e00000011e248000  IN   0.2  554496  21456  crsd.bin
  11892      1   0  e000000144178000  IN   0.2  554496  21456  crsd.bin
  11917      1   1  e0000001ba4c8000  IN   0.2  554496  21456  crsd.bin
  12125      1   3  e000004072e70000  IN   0.2  554496  21456  crsd.bin
  12160      1   1  e000000166d40000  IN   0.2  554496  21456  crsd.bin
  12229      1   1  e000000239588000  IN   0.1   49024  11264  extproc
  12238      1   1  e000000122090000  IN   0.2  554496  21456  crsd.bin
  12252      1   2  e000004049388000  IN   0.2  554496  21456  crsd.bin
  12253      1   2  e0000040c5b68000  IN   0.2  554496  21456  crsd.bin
  12254      1   1  e0000001c9380000  IN   0.2  554496  21456  crsd.bin
  12255      1   1  e0000040c74a0000  IN   0.2  554496  21456  crsd.bin
  12256      1   1  e000000198670000  IN   0.2  554496  21456  crsd.bin
  12257      1   1  e0000040b0b68000  IN   0.2  554496  21456  crsd.bin
  12258      1   1  e00000000e1f0000  IN   0.2  554496  21456  crsd.bin
  13315      1   1  e0000001a7e28000  IN  26.1 5025840 2722752  oracle
  13478      1   0  e0000040a3518000  IN   0.1   49024  11264  extproc
  14292      1   1  e00000019a530000  IN   0.2  554496  21456  crsd.bin
  14329      1   0  e00000404b580000  IN   0.2  554496  21456  crsd.bin
  14336      1   3  e0000001c1018000  IN   0.2  554496  21456  crsd.bin
  14519  32743   0  e0000040a9360000  IN   0.1  199888  11408  ocssd.bin
  14520  32743   1  e0000001f78e8000  IN   0.1  199888  11408  ocssd.bin
  14689      1   0  e0000040f1808000  IN   0.2  554496  21456  crsd.bin
  14805      1   0  e0000002795d8000  IN   0.0    2976    992  acpid
  14816      1   3  e000000278cb0000  IN   0.1   72768  10320  snmpd
  14858      1   2  e000000278020000  IN   0.0   10448   2064  sshd
  14975      1   3  e000000101600000  IN   0.0    5792   1904  xinetd
  14988      1   0  e000000278610000  IN   0.1   10800  12672  ntpd
  14998      1   1  e0000040f9f38000  IN   0.0    9824   1888  vsftpd
  15017      1   2  e0000040f8ea0000  IN   0.0   18208   2832  sendmail
  15026      1   2  e000000279218000  IN   0.0   15680   2128  sendmail
  15073      1   0  e00000027bcf0000  IN   0.0   77120   1824  hpsmhd
  15074  15073   0  e00000027b918000  IN   0.0    9488   1168  rotatelogs
  15075  15073   0  e00000027aae0000  IN   0.0    9488   1184  rotatelogs
  15086      1   2  e00000027bfa8000  IN   0.0   55872   1600  crond
  15097  15073   0  e000000104340000  IN   0.0  353840   1376  hpsmhd
  15099  15073   2  e0000001044c0000  IN   0.0  353840   1376  hpsmhd
  15100  15073   0  e0000001044c8000  IN   0.0  353840   1376  hpsmhd
  15101  15073   0  e0000001044e0000  IN   0.0  353840   1376  hpsmhd
  15102  15073   0  e0000001044e8000  IN   0.0  353840   1376  hpsmhd
  15103  15073   0  e000000104500000  IN   0.0  353840   1376  hpsmhd
  15104  15073   0  e000000104508000  IN   0.0  353840   1376  hpsmhd
  15105  15073   0  e000000104520000  IN   0.0  353840   1376  hpsmhd
  15106  15073   0  e000000104528000  IN   0.0  353840   1376  hpsmhd
  15107  15073   0  e000000104540000  IN   0.0  353840   1376  hpsmhd
  15108  15073   0  e000000104548000  IN   0.0  353840   1376  hpsmhd
  15109  15073   0  e000000104560000  IN   0.0  353840   1376  hpsmhd
  15110  15073   0  e000000104568000  IN   0.0  353840   1376  hpsmhd
  15111  15073   0  e000000104570000  IN   0.0  353840   1376  hpsmhd
  15112  15073   0  e000000104588000  IN   0.0  353840   1376  hpsmhd
  15113  15073   0  e000000104590000  IN   0.0  353840   1376  hpsmhd
  15114  15073   0  e0000001045a8000  IN   0.0  353840   1376  hpsmhd
  15115  15073   0  e0000001045b0000  IN   0.0  353840   1376  hpsmhd
  15116  15073   0  e0000001045c8000  IN   0.0  353840   1376  hpsmhd
  15117  15073   0  e0000001045d0000  IN   0.0  353840   1376  hpsmhd
  15118  15073   0  e0000001045e8000  IN   0.0  353840   1376  hpsmhd
  15119  15073   0  e0000001045f0000  IN   0.0  353840   1376  hpsmhd
  15120  15073   0  e000000104608000  IN   0.0  353840   1376  hpsmhd
  15121  15073   0  e000000104610000  IN   0.0  353840   1376  hpsmhd
  15122  15073   0  e000000104628000  IN   0.0  353840   1376  hpsmhd
  15123  15073   0  e000000104640000  IN   0.0  353840   1376  hpsmhd
  15124  15073   0  e000000104648000  IN   0.0  353840   1376  hpsmhd
  15157      1   3  e000000104c48000  IN   0.1   82240   9328  hpimaX
  15166      1   0  e0000001055e0000  IN   0.0   70064   2480  hpimafdtnpeerd
  15168      1   3  e000000105718000  IN   0.0   70064   2480  hpimafdtnpeerd
  15176      1   0  e0000001058a0000  IN   0.1   62544   6912  hpimahostd
  15185      1   3  e000000105aa8000  IN   0.0   59744   2400  hpimathreshd
  15194      1   0  e000000105d60000  IN   0.0   70288   3392  hpimasvrpeerd
  15238      1   0  e000000105348000  IN   0.0    9168   2672  hpimahealthd
  15247      1   0  e000000105ba0000  IN   0.0   10032   3776  hpimastdeqd
  15256      1   2  e00000027a158000  IN   0.0   70432   3216  hpimastorpeerd
  15258      1   3  e000000105f88000  IN   0.0   70288   3392  hpimasvrpeerd
  15259      1   1  e000000106380000  IN   0.0   70432   3216  hpimastorpeerd
  15260      1   0  e000000106480000  IN   0.1   82240   9328  hpimaX
  15269      1   0  e000000107938000  IN   0.0   31968   2880  hpimaeventd
  15318      1   0  e000000107978000  UN   0.0   32176   4000  hpimaidad
  15322      1   3  e000000106d18000  IN   0.0   32176   4000  hpimaidad
  15330      1   0  e000000107d28000  IN   0.0    9008   1344  hpimasasd
  15384      1   1  e000000039028000  IN   0.0    9008   1552  hpimascsid
  15422      1   3  e000000039b70000  IN   0.0   32176   3968  hpimafcad
  15477      1   1  e0000000381d8000  IN   0.0   70192   2768  hpimanicpeerd
  15501      1   3  e000000106438000  IN   0.0   70192   2768  hpimanicpeerd
  15604      1   3  e000000039b30000  IN   0.0   32176   3968  hpimafcad
  15650      1   1  e00000003aa48000  IN   0.0    9536   2496  hpimanicd
  15797      1   2  e000000250480000  IN  25.2 5025840 2629152  oracle
  16850      1   0  e0000001d1500000  IN   0.2  554496  21456  crsd.bin
  17164      1   2  e0000001bb2b8000  IN   0.3 4684192  27488  oracle
  17166      1   1  e000000142f18000  IN   0.2 4690480  18912  oracle
  17168      1   2  e0000001ae680000  RU   0.6 4690736  57744  oracle
  17170      1   2  e000004057bf0000  RU   0.8 4697536  87632  oracle
  17172      1   2  e00000011ce48000  RU  20.8 4697552 2172304  oracle
  17174      1   2  e000004042d68000  RU  22.8 4697552 2381392  oracle
  17176      1   2  e00000407afb8000  IN   0.2 4682912  16224  oracle
  17178      1   3  e00000013e498000  IN  29.1 4692800 3038192  oracle
  17180      1   3  e00000014b1e0000  IN  29.4 4691520 3065216  oracle
  17182      1   1  e0000040fe4f0000  IN   0.4 4705584  42784  oracle
  17184      1   3  e000000210e00000  IN   2.7 4690624 282608  oracle
  17186      1   2  e00000012a810000  IN   5.3 4689344 552656  oracle
  17188      1   2  e0000002551c8000  IN   0.5 4685152  48592  oracle
  17190      1   2  e00000011d578000  IN   4.5 4687456 464992  oracle
  17192      1   3  e00000026b528000  IN   0.2 4684208  15984  oracle
  17194      1   2  e0000040a2600000  IN   0.3 4684208  26640  oracle
  17196      1   2  e000000027760000  IN   0.3 4684208  26640  oracle
  17198      1   1  e0000002274e0000  IN   0.3 4684208  27536  oracle
  17200      1   2  e000000030e90000  IN   0.3 4684208  27408  oracle
  17202      1   1  e0000001f02a0000  IN   0.2 4684208  24544  oracle
  17204      1   0  e0000001be5e0000  IN   1.4 4686544 148432  oracle
  17206      1   2  e000000239a68000  IN   1.5 4686480 156048  oracle
  17208      1   0  e000000011198000  IN   1.0 4686384 105792  oracle
  17217      1   2  e0000001d3798000  IN   1.0 4686384 106112  oracle
  17218      1   1  e0000040fbd78000  IN   0.0   70832   2768  hpimaE0traps
  17226      1   0  e0000040dc740000  IN   0.7 4686384  71216  oracle
  17228      1   3  e00000018ebe8000  IN   0.6 4686384  67520  oracle
  17243      1   3  e0000001bcc08000  IN   2.4 4690592 247216  oracle
  17263      1   0  e0000040f2ff0000  IN   0.3 4685488  34736  oracle
  17271      1   3  e000000195f40000  IN   0.1  492720   8864  oracle
  17273      1   3  e000000221368000  IN   0.2 4685472  18544  oracle
  17304      1   3  e0000040f6218000  IN   0.0   48208    784  salinfod
  17307      1   0  e0000040f5c80000  IN   0.0   48208    784  salinfod
  17308      1   3  e00000003ac48000  IN   0.0   48208    784  salinfod
  17309      1   3  e000000107d20000  IN   0.0   48208    784  salinfod
  17310      1   0  e00000003be68000  IN   0.0   48208    784  salinfod
  17311      1   3  e000004054d18000  IN   0.1  102736   7072  racgimon
  17313      1   1  e0000001b5110000  IN   0.1  102736   7072  racgimon
  17415      1   3  e0000040f6170000  IN   0.0    5488   1120  dbus-daemon-1
  17444      1   2  e000000104008000  IN   0.0   53248    992  rhnsd
  17476      1   0  e000000039a18000  IN   0.3   32592  30368  hald
  17555      1   0  e000000132b30000  IN   0.1  102736   7072  racgimon
  17556      1   2  e0000001720d8000  IN   0.1  102736   7072  racgimon
  17558      1   2  e000000103e50000  IN   0.0    3024    992  mingetty
  17559      1   2  e000000279c18000  IN   0.0    3024    992  mingetty
  17560      1   2  e0000040f6a38000  IN   0.0    3024    992  mingetty
  17561      1   2  e000000278b18000  IN   0.0    3024    992  mingetty
  17562      1   0  e000000106ca8000  IN   0.0    3024    992  mingetty
  17563      1   2  e00000003b5c0000  IN   0.0    3024    992  mingetty
  17564      1   1  e0000040f3648000  IN   0.0    6432   3232  utild
  17785      1   3  e000004041b08000  IN   0.3 4699488  28896  oracle
> 17811      1   2  e000000214cc0000  RU   6.7 4992608 698080  oracle
  17813      1   2  e00000025f9d8000  RU   6.9 4992608 720272  oracle
  17815      1   3  e000004080e28000  RU   5.6 4910704 581712  oracle
  17817      1   2  e000000115138000  IN   3.7 4845152 390752  oracle
  17819      1   2  e00000025a518000  RU   3.8 4845152 397024  oracle
  17824      1   3  e0000001b5b10000  RU   4.7 4877936 495024  oracle
  17834      1   2  e000004052bd8000  IN   0.8 4707712  79696  oracle
  17837      1   1  e0000001f9ad8000  RU   4.5 4861536 466992  oracle
  17855  15086   2  e0000001f9888000  IN   0.0   57968   3568  crond
  17885  14858   3  e000000116458000  IN   0.1   14512   7392  sshd
  17887  17885   0  e000004068930000  IN   0.1   14512   5888  sshd
  17888  17887   3  e000004096128000  IN   0.0   55504   4240  bash
  18017      1   2  e000000122200000  IN   0.1  102736   7072  racgimon
  18140      1   3  e0000001b4ec0000  IN   0.1  102736   7072  racgimon
  18166      1   2  e00000404aba8000  IN  12.9 4687408 1340832  oracle
  18181      1   2  e0000001c5028000  IN   2.2 4687408 227024  oracle
  18395      1   3  e000004042830000  IN  33.9 5107696 3532848  oracle
  18402      1   3  e000000106e08000  IN  33.2 5107696 3461248  oracle
  18415      1   1  e000000145c68000  IN  20.8 5058544 2166720  oracle
  18423      1   1  e00000409cb68000  IN  20.5 5058544 2132880  oracle
  18427      1   2  e0000001e09f0000  IN  33.2 5107744 3460880  oracle
  18429      1   2  e0000002175b0000  RU   1.6 4690656 163888  oracle
  18432      1   0  e00000002fdc8000  IN  33.1 5107712 3454272  oracle
  18437      1   2  e0000040d9f48000  IN  33.1 5107696 3453296  oracle
  18756  18572   3  e0000040dfa68000  IN   0.0   54464   1904  sleep
  19051      1   3  e000000213930000  IN   0.2 4686512  23840  oracle
  19072      1   3  e00000021fe00000  IN   0.2 4683024  23680  oracle
  19172      1   1  e0000040a10e0000  IN   6.6 4731760 689168  oracle
  19188      1   2  e00000021c4b0000  IN   2.4 4684320 251424  oracle
  19245      1   3  e0000001bc460000  IN  30.2 4722000 3146624  oracle
  19252      1   2  e000000017658000  IN  30.6 4722000 3194368  oracle
  19576  30266   1  e0000040d3278000  IN   0.0    4976   3536  init.cssd
  19578     10   3  e00000018dfa0000  IN   0.0       0      0  [pdflush]
  19611  19576   2  e0000040556d0000  IN   0.0    4976   3568  init.cssd
  19622  19611   3  e00000023dbc0000  RU   0.0    4976   2224  init.cssd
  20711      1   0  e0000001a7278000  IN   0.1   41232   8896  tnslsnr
  20846      1   3  e000000120bc0000  IN   0.1   49024  11264  extproc
  21240      1   2  e00000001ded0000  IN   7.7 5009104 799072  oracle
  21542      1   0  e0000040612b0000  IN   9.8 4687472 1016848  oracle
  21545      1   2  e0000001b04a8000  IN  25.5 5025840 2654352  oracle
  21661      1   0  e000000014300000  IN   0.1   49024  11264  extproc
  21672      1   0  e00000011fee0000  IN   0.1   49024  11264  extproc
  21677      1   3  e0000040dac28000  IN   0.1   49024  11264  extproc
  21688      1   3  e000000264d58000  IN   0.1   49024  11264  extproc
  21695      1   1  e0000040b64f8000  IN   0.1   49024  11264  extproc
  21784      1   1  e000000039070000  IN   0.1   49024  11264  extproc
  21938  30265   3  e0000040eec98000  IN   0.1  180496  10768  evmd.bin
  22807      1   0  e0000040fe548000  IN   0.0    3040   1136  agetty
  23595  14858   2  e0000040aeb78000  IN   0.0   14512   4288  sshd
  23596  14858   1  e0000001c4998000  IN   0.0   14512   4288  sshd
  23628  23595   2  e00000408ddd8000  IN   0.0   14512   3504  sshd
  23629  23596   2  e000000015690000  IN   0.0   14512   3504  sshd
  23630  23628   0  e000004099808000  IN   0.0   55296   3232  bash
  23663  23629   2  e0000001ae460000  IN   0.0   55296   3232  bash
  23867      1   2  e00000021e0f0000  IN   0.1   49024  11264  extproc
  24628      1   2  e0000040d1db8000  IN  26.6 5009456 2770768  oracle
  24636     10   0  e000000027630000  IN   0.0       0      0  [pdflush]
  25008      1   0  e00000011c5e8000  IN   0.2   49024  15872  extproc
  25685  14858   2  e0000001d9f20000  IN   0.1   14512   7392  sshd
  25700  25685   2  e000000150f58000  IN   0.1   14512   5952  sshd
  25701  25700   1  e0000002740a8000  IN   0.0   55504   4336  bash
  25808  14858   0  e000004097b98000  IN   0.1   14512   7392  sshd
  25810  25808   2  e0000002385f8000  IN   0.1   14512   5952  sshd
  26154      1   2  e000000154788000  IN   0.2 4683024  23088  oracle
  27645      1   2  e00000012a508000  IN   0.1   49024  11264  extproc
  27838      1   2  e000004088710000  IN   2.4 4687392 247616  oracle
  30266      1   3  e000000192a50000  IN   0.0    4976   2992  init.cssd
  31322      1   2  e0000000226a0000  IN  14.3 4687424 1485904  oracle
  32690  30266   3  e000004093fe8000  IN   0.0    4976   2352  init.cssd
  32695  30265   0  e0000001adfa8000  IN   0.1  180496  10768  evmd.bin


===========================
STACK TRACE OF FAILING TASK
===========================

PID: 7169   TASK: e00000027ec30000  CPU: 0   COMMAND: "kipmi0"
 #0 [BSP:e00000027ec31210] start_disk_dump at a0000002008f1170
 #1 [BSP:e00000027ec311e8] try_crashdump at a0000001000b02d0
 #2 [BSP:e00000027ec311a8] die at a00000010003e900
 #3 [BSP:e00000027ec31178] die_if_kernel at a00000010003ea20
 #4 [BSP:e00000027ec31150] ia64_bad_break at a00000010003ebc0
 #5 [BSP:e00000027ec31150] ia64_leave_kernel at a00000010000f600
  EFRAME: e00000027ec37a90
      B0: a0000001000d2c50      CR_IIP: a0000001000d2c50
 CR_IPSR: 0000101008022018      CR_IFS: 8000000000000207
  AR_PFS: 0000000000000207      AR_RSC: 0000000000000003
 AR_UNAT: 0000000000000000     AR_RNAT: 0000000000000000
  AR_CCV: 0000000000000004     AR_FPSR: 0009804c8a70033f
  LOADRS: 0000000000000000 AR_BSPSTORE: a000000200abc000
      B6: a0000001000724e0          B7: a00000010045c720
      PR: 000a06a60002a595          R1: a0000001009bb9e0
      R2: e000000001004ab8          R3: e000000001010000
      R8: 0000000000000020          R9: 0000000000000000
     R10: a0000001007d7050         R11: a0000001007d7050
     R12: e00000027ec37c50         R13: e00000027ec30000
     R14: 0000000000004000         R15: a00000010074fbc0
     R16: a00000010074fbc8         R17: e000000101e6fde8
     R18: 0000000000000001         R19: 0000000000000004
     R20: 0000000000000004         R21: 0000000000000004
     R22: 0000000000000001         R23: 0000000000000000
     R24: 0000000000000000         R25: 0000000000000004
     R26: e00000027ec30dd0         R27: 0000000000000074
     R28: 0000000000000074         R29: 0000000000000065
     R30: e000000101e68050         R31: 00000000356f424c
      F6: 1003e0000000000001200     F7: 1003e8080808080808081
      F8: 1003e00000000000023dc     F9: 1003e000000000e580000
     F10: 1003e00000000356f424c    F11: 1003e44b831eee7285baf
 #6 [BSP:e00000027ec31118] unlock_page at a0000001000d2c50
 #7 [BSP:e00000027ec310e0] mpage_end_io_read at a00000010017a8b0
 #8 [BSP:e00000027ec310a8] bio_endio at a000000100133110
 #9 [BSP:e00000027ec30fb8] do_cciss_intr at a0000002000c5020
#10 [BSP:e00000027ec30f78] handle_IRQ_event at a0000001000130f0
#11 [BSP:e00000027ec30f10] do_IRQ at a000000100013c30
#12 [BSP:e00000027ec30ec8] ia64_handle_irq at a000000100015d10
#13 [BSP:e00000027ec30ec8] ia64_leave_kernel at a00000010000f600
  EFRAME: e00000027ec37c60
      B0: a0000002008be6e0      CR_IIP: a0000001000aea00
 CR_IPSR: 0000101008126010      CR_IFS: 8000000000000002
  AR_PFS: 000000000000048b      AR_RSC: 0000000000000003
 AR_UNAT: 0000000000000000     AR_RNAT: 0000000000000000
  AR_CCV: 0000000000000000     AR_FPSR: 0009804c8a70433f
  LOADRS: 0000000000000000 AR_BSPSTORE: 0000000000000000
      B6: a00000010025f420          B7: a0000002008bffa0
      PR: 0000000000006641          R1: a0000001009bb9e0
      R2: 0009ca9292346e96          R3: 0000000000000001
      R8: 0000000000000001          R9: 00000000004c2520
     R10: 00000000004c2520         R11: a0000001007d2750
     R12: e00000027ec37e20         R13: e00000027ec30000
     R14: 0000000000000646         R15: 0000000000000640
     R16: 0009ca9292346850         R17: a0000002008ca1b0
     R18: a0000002008bffa0         R19: e0000040ffb0c4c4
     R20: 0000000000000001         R21: 0000000000000006
     R22: 0000000000000001         R23: 0000000000000005
     R24: 0000000000000005         R25: a0000002008ca1b0
     R26: a0000002008bffa0         R27: 0000001008126010
     R28: e0000040ffb0c405         R29: e00000003fe103e8
     R30: 0000000000000000         R31: e00000003fe103a4
      F6: 1003e0000000000000001     F7: 1003e0000000000000001
      F8: 1003e0000000000000001     F9: 1003e000000000fb7f627
     F10: 1003e000000003a8e6ace    F11: 1003e44b831eee7285baf
#14 [BSP:e00000027ec30eb8] kthread_should_stop at a0000001000aea00
#15 [BSP:e00000027ec30e70] ipmi_thread at a0000002008be6e0
#16 [BSP:e00000027ec30e38] kthread at a0000001000aec20
#17 [BSP:e00000027ec30e10] kernel_thread_helper at a000000100018c70
#18 [BSP:e00000027ec30e10] start_kernel_thread at a000000100008c60


How reproducible:

Unknown.
Comment 1 Luming Yu 2007-08-26 21:18:44 EDT
Is it same issue with bz# 248954.
Please test this patch : https://bugzilla.redhat.com/show_bug.cgi?id=248954#c1
Comment 2 Eduardo Alvarenga 2007-08-26 21:50:08 EDT
bz# 248954 says that it is reproducible on ASM startup, but this bug occurs on
an already running ASM/DB instance. 

Are you sure this is really related to bz# 248954? The ASMlib, DB and kernel
versions differ and I cannot see no ia64_spinlock_contention on the kernel
backtrace. Is this related to bio?
Comment 3 Luming Yu 2007-08-27 01:14:15 EDT
I'm __not__ sure, but it just sounds relative bugs. And probably you need to
test a kernel with patch at bz# 252400 too. 
Comment 4 Eduardo Alvarenga 2007-08-27 01:22:13 EDT
Does 2.6.9-56.EL already include both patches? Will v4.6 be based on this
kernel? Any release date?
Comment 5 Luming Yu 2007-08-27 01:28:35 EDT
Please just check the 2.6.9-56 kernel source to figure out if those patches
included. If __not__, please try apply those patches and re-test.

Comment 6 Luming Yu 2007-08-28 03:56:11 EDT
To comment$ 4, is the problem resolved with the two patches?
Comment 7 Eduardo Alvarenga 2007-08-29 17:37:22 EDT
I have no way to test both patches on this specific system. This is a mission 
critical production machine. I have also a case opened on HP, but they are 
taking too long to provide a solution for this problem. 

That´s why I filled a bug report on bugzilla hoping to get a faster response. 
HP suggests to upgrade to version 2.6.9-56 (pre v4.6) but this is not yet 
supported either by HP or RedHat, so that´s why we can not run those tests.

One thing that could be done is to backport those patches to supported RedHat 
update so we can test it.
Comment 8 Luming Yu 2007-08-29 23:07:48 EDT
Please help get as much information as possible that are necessary to re-produce
the problem... if you think it is __not__ suitable on bugziila, you can direclty
send the info to my email account: luyu@redhat.com 
Comment 9 Eduardo Alvarenga 2007-08-29 23:30:23 EDT
Can you provide FTP access so I can send you the vmcore? If so please send me 
the access details to my email eduardo.alvarenga () gmail.com
Comment 10 Luming Yu 2007-08-29 23:46:15 EDT
yes, vmcore is helpful to analyze the problem. To fix the problem, it is also
very  useful if the problem can be re-produced. So please help me get
information such as the machine type, the os version, the configuation, the
workloads,.... that others can try to use to reproduce the problem...

Comment 11 Prarit Bhargava 2007-09-04 10:15:53 EDT
Luming, the requested info is in comment #1.

Adding dchapman, HP on-site engineer @ RH.

P.
Comment 12 Luming Yu 2007-09-27 08:57:29 EDT
Please update the status of the bug, I'd like to know if the bug is just caused
by firmware/memory. If the problem still exists, I would try my best to
re-produce it. So far I'm not able to reproduce it.
Comment 13 Luming Yu 2007-10-23 10:33:19 EDT
Based on comments :https://bugzilla.redhat.com/show_bug.cgi?id=261121#c34
https://bugzilla.redhat.com/show_bug.cgi?id=261121#c35
https://bugzilla.redhat.com/show_bug.cgi?id=261121#c36
https://bugzilla.redhat.com/show_bug.cgi?id=261121#c41,

the problem is a HW or firmware issue withthe OEM's box. Because: 1. It is
__not__ reproducible with other box. 2. After CPU and memory update, I am not
aware of any panics since the update. So this is __not__ kernel BUG. Resolve
bug, changing status to __not_ a bug. If I'm wrong, please re-open it.

Note You need to log in before you can comment on or make changes to this bug.