Description of problem: Kernel Panic CPU: 3 EIP: 0060:[<c02d4cab>] Tainted: P VLI EFALGS: 00010002 (2.6.9-55.ELsmp) EIP is at _spin_unlock_irg+0x1c/0x28 eax: 00000001 ebx: 00000075 ecx: 01bf1601 edx: c0328b4c esi: 00000000 edi: f6c97f4c ebp: 00000fff esp: f6c97f24 ds: 007b es: 007b ss: 0068 Process klogd (pid:6047, threadinfo=f6c97000 task=f53461b0) Stack: c012242f 00000000 00000000 00000064 00ff1704 00000000 f53461b0 c012052d f6c97f64 f6c97f64 00000000 00000000 00000000 00000000 f53461b0 c012052d f6c97f64 f6c97f64 00000000 00000000 0000006e c0331bc0 f5960180 00000fff Call Trace: [<c012242f>] do_syslog+0x183/0x3bf [<c012052d>] autoremove_wake_function+0x0/0x2d [<c012052d>] autoremove_wake_function+0x0/0x2d [<c015b0ce>] vfs_read+0xb6/0xe2 [<c015b2e3>] sys_read+0x3c/0x62 [<c02d5ee3>] syscall_call+0x7/0xb [<c02d007b>] unix_dgram_recvmsg+0x11/0x1da Code: 08 0f 0b 6d 00 37 75 2e c0 86 11 53 9d 5b c3 81 78 04 ad 4e ad de 89 c2 b1 01 74 08 0f 0b 6c 00 37 75 2e c0 0f b6 02 84 c0 7e 08 <0f> 0b 6d 00 37 75 2e c0 86 0a fb c3 81 78 04 ad 4e ad de 89 c2 <0>Fatal exception: panic in 5 seconds Kernel panic - no syncing: Fatal exception Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Hello, We need a lot more information to diagnose this problem. Could you please provide a sysreport for the system? Also what was the system doing before it panicked? Thanks, P.
Created attachment 291716 [details] sysreport from system
This is a HP DL380 running HP ServiceGuard clustering in a single node configuration. It run an Oracle DB version 10.2.02. It is dual connected to an EMC SAN using EMC PowerPath to manage the connections. This system also serves as a DR system, so there are disks that are masked and zoned to the system but are not enabled for access until needed for DR. I am not aware of anything unusual going on the system at the time of the crash. The last few syslog file entries are unusual though. Jan 11 12:24:00 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344156), 0 Jan 11 12:24:04 hsg09lt kernel: audit(1200075844.263:21739): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-2 ino=2 scontext=user_u:system_r:snmpd_t tcontex t=system_u:object_r:tmp_t tclass=dir Jan 11 12:24:04 hsg09lt kernel: audit(1200075844.313:21740): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-12 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:04 hsg09lt kernel: audit(1200075844.324:21741): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-13 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:04 hsg09lt kernel: audit(1200075844.338:21742): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-14 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:04 hsg09lt kernel: audit(1200075844.347:21743): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-15 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344158), 0 Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344152), 0 Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344155), 0 Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344154), 0 Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344160), 0 Jan 11 12:24:36 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (344159), 0 Jan 11 12:24:37 hsg09lt kernel: audit(1200075877.026:21744): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-2 ino=2 scontext=user_u:system_r:snmpd_t tcontex t=system_u:object_r:tmp_t tclass=dir Jan 11 12:24:37 hsg09lt kernel: audit(1200075877.061:21745): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-12 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:37 hsg09lt kernel: audit(1200075877.068:21746): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-13 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:37 hsg09lt kernel: audit(1200075877.075:21747): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-14 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:24:37 hsg09lt kernel: audit(1200075877.084:21748): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-15 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:25:15 hsg09lt kernel: audit(1200075915.297:21749): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-2 ino=2 scontext=user_u:system_r:snmpd_t tcontex t=system_u:object_r:tmp_t tclass=dir Jan 11 12:25:15 hsg09lt kernel: audit(1200075915.335:21750): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-12 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:25:15 hsg09lt kernel: audit(1200075915.347:21751): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-13 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:25:15 hsg09lt kernel: audit(1200075915.362:21752): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-14 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:25:15 hsg09lt kernel: audit(1200075915.369:21753): avc: denied { getattr } for pid=6385 comm="snmpd" name="/" dev=dm-15 ino=2 scontext=user_u:system_r:snmpd_t tconte xt=system_u:object_r:file_t tclass=dir Jan 11 12:25:32 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (721306), 0 Jan 11 12:25:32 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (721309), 0 Jan 11 12:25:32 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (721307), 0 Jan 11 12:25:32 hsg09lt kernel: EXT3-fs warning (device dm-12): ext3_unlink: Deleting nonexistent file (721308), 0
Created attachment 310046 [details] debug patch Pretty sure I know what this is but I'd like to confirm first. Would you run with this debug patch and give me logs if you start getting backtraces in your logs. Thanks much.