Bug 594604 - Kernel panic, "kernel BUG at mm/slab.c:2974!"
Summary: Kernel panic, "kernel BUG at mm/slab.c:2974!"
Keywords:
Status: CLOSED DUPLICATE of bug 585926
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel
Version: 6.0
Hardware: x86_64
OS: Linux
low
high
Target Milestone: rc
: ---
Assignee: Red Hat Kernel Manager
QA Contact: Red Hat Kernel QE team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-05-21 07:07 UTC by Taunus
Modified: 2010-07-20 00:55 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-07-20 00:55:46 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Taunus 2010-05-21 07:07:28 UTC
Description of problem:
Kernel panic, "kernel BUG at mm/slab.c:2974!"

Version-Release number of selected component (if applicable):
rhel6 beta

How reproducible:
This on occured same time after launching a java app that consumes about 1gb memory.

Steps to Reproduce:
1. launching a java app that consumes about 1gb memory
2. 
3.
  
Actual results:
kernel panic

Expected results:
no kernel panic

Additional info:
Intel(R) Core2 Duo CPU
ATI Mobility Radeon HD 3470


------------[ cut here ]------------
kernel BUG at mm/slab.c:2974!
invalid opcode: 0000 [#1] SMP 
last sysfs
file: /sys/devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda/sda1/stat
CPU 0 
Modules linked in: netconsole(U) configfs(U) ipt_MASQUERADE(U)
iptable_nat(U) nf_nat(U) rfcomm(U) sco(U) bridge(U) stp(U) llc(U)
bnep(U) l2cap(U) autofs4(U) nfs(U) lockd(U) fscache(U) nfs_acl(U)
auth_rpcgss(U) sunrpc(U) cpufreq_ondemand(U) acpi_cpufreq(U)
freq_table(U) xt_physdev(U) ip6t_REJECT(U) nf_conntrack_ipv6(U)
ip6table_filter(U) ip6_tables(U) ipv6(U) dm_mirror(U) dm_region_hash(U)
dm_log(U) kvm(U) uinput(U) snd_hda_codec_conexant(U) snd_hda_intel(U)
snd_hda_codec(U) arc4(U) ecb(U) snd_hwdep(U) iwlagn(U) snd_seq(U)
snd_seq_device(U) iwlcore(U) uvcvideo(U) snd_pcm(U) zaurus(U)
videodev(U) snd_timer(U) mac80211(U) iTCO_wdt(U) e1000e(U)
thinkpad_acpi(U) v4l1_compat(U) cdc_ether(U) ppdev(U) cdc_acm(U)
v4l2_compat_ioctl32(U) btusb(U) iTCO_vendor_support(U) usbnet(U) snd(U)
hwmon(U) i2c_i801(U) mii(U) joydev(U) cfg80211(U) sr_mod(U)
parport_pc(U) bluetooth(U) soundcore(U) sg(U) parport(U) cdc_wdm(U)
wmi(U) cdrom(U) snd_page_alloc(U) rfkill(U) ext4(U) mbcache(U) jbd2(U)
cryptd(U) aes_x86_64(U) aes_generic(U) xts(U) gf128mul(U) dm_crypt(U)
dm_multipath(U) sd_mod(U) crc_t10dif(U) yenta_socket(U)
rsrc_nonstatic(U) video(U) output(U) firewire_ohci(U) firewire_core(U)
crc_itu_t(U) ahci(U) radeon(U) ttm(U) drm_kms_helper(U) drm(U)
i2c_algo_bit(U) i2c_core(U) dm_mod(U) [last unloaded: scsi_wait_scan]
Pid: 18910, comm: ksmtuned Not tainted 2.6.32-19.el6.x86_64 #1 2768Z28
RIP: 0010:[<ffffffff8114938c>]  [<ffffffff8114938c>] cache_alloc_refill
+0x1ec/0x240
RSP: 0018:ffff880071a33cc0  EFLAGS: 00010046
RAX: 0000000000000016 RBX: ffff88013bf10ac0 RCX: ffff880066c62df8
RDX: ffff88010f77f000 RSI: 0000000000000090 RDI: ffff880066c62000
RBP: ffff880071a33d10 R08: ffff88010f77f000 R09: 0000000000000000
R10: 0000000000013510 R11: ffff880131dc7cac R12: ffff88013bec7400
R13: ffff88013be7d9c0 R14: 0000000000000016 R15: ffff880066c62000
FS:  00007f40c70f5700(0000) GS:ffff880028200000(0000)
knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000097acf0 CR3: 0000000071a22000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process ksmtuned (pid: 18910, threadinfo ffff880071a32000, task
ffff880033a22080)
Stack:
 0000003a00000016 000412d000000000 ffff88013be7d9e0 ffff88013be7d9d0
<0> ffff880071a33d20 ffff880071b34ad8 00000000000000d0 ffff88013bf10ac0
<0> 00000000000000d0 0000000000000246 ffff880071a33d50 ffffffff8114a288
Call Trace:
 [<ffffffff8114a288>] kmem_cache_alloc+0x1c8/0x220
 [<ffffffff81132b61>] anon_vma_fork+0x51/0xa0
 [<ffffffff81065592>] dup_mm+0x232/0x4d0
 [<ffffffff810665d1>] copy_process+0xd31/0x1380
 [<ffffffff81066de4>] do_fork+0x94/0x480
 [<ffffffff8112a6be>] ? handle_mm_fault+0x1ee/0x2b0
 [<ffffffff810cb172>] ? audit_syscall_entry+0x242/0x270
 [<ffffffff81011548>] sys_clone+0x28/0x30
 [<ffffffff81013493>] stub_clone+0x13/0x20
 [<ffffffff81013172>] ? system_call_fastpath+0x16/0x1b
Code: 5c 41 5d 41 5e 41 5f c9 c3 66 0f 1f 44 00 00 49 8b 55 10 48 8b 75
c8 4c 89 ff e8 90 ba 10 00 8b 45 b0 e9 79 ff ff ff 0f 0b eb fe <0f> 0b
66 90 eb fc 8b 55 b8 8b 75 bc 31 c9 48 89 df e8 ce f8 ff 
RIP  [<ffffffff8114938c>] cache_alloc_refill+0x1ec/0x240
 RSP <ffff880071a33cc0>
---[ end trace 167a156646f2a8bd ]---
Kernel panic - not syncing: Fatal exception
Pid: 18910, comm: ksmtuned Tainted: G      D    2.6.32-19.el6.x86_64 #1
Call Trace:
 [<ffffffff814bfd69>] panic+0x78/0x137
 [<ffffffff814c3d1c>] oops_end+0xdc/0xf0
 [<ffffffff8101723b>] die+0x5b/0x90
 [<ffffffff814c35c4>] do_trap+0xc4/0x160
 [<ffffffff81014cb5>] do_invalid_op+0x95/0xb0

Comment 2 Pekka Järveläinen 2010-05-24 06:45:02 UTC
it happens about every third time launching liferay-portal 6.0.1 (which starts
also firefox) when screen is locked:

------------[ cut here ]------------
kernel BUG at mm/slab.c:2974!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda/sda1/stat
CPU 0 
Modules linked in: netconsole(U) configfs(U) ipt_MASQUERADE(U) iptable_nat(U) nf_nat(U) rfcomm(U) sco(U) bridge(U) stp(U) llc(U) bnep(U) l2cap(U) autofs4(U) nfs(U) lockd(U) fscache(U) nfs_acl(U) auth_rpcgss(U) sunrpc(U) cpufreq_ondemand(U) acpi_cpufreq(U) freq_table(U) xt_physdev(U) ip6t_REJECT(U) nf_conntrack_ipv6(U) ip6table_filter(U) ip6_tables(U) ipv6(U) dm_mirror(U) dm_region_hash(U) dm_log(U) kvm(U) uinput(U) snd_hda_codec_conexant(U) snd_hda_intel(U) snd_hda_codec(U) arc4(U) snd_hwdep(U) ecb(U) ppdev(U) snd_seq(U) parport_pc(U) iwlagn(U) snd_seq_device(U) thinkpad_acpi(U) snd_pcm(U) hwmon(U) parport(U) iwlcore(U) mac80211(U) snd_timer(U) zaurus(U) wmi(U) uvcvideo(U) btusb(U) sr_mod(U) snd(U) cdc_ether(U) sg(U) videodev(U) cdrom(U) e1000e(U) cdc_acm(U) cfg80211(U) v4l1_compat(U) usbnet(U) bluetooth(U) i2c_i801(U) soundcore(U) mii(U) v4l2_compat_ioctl32(U) iTCO_wdt(U) cdc_wdm(U) rfkill(U) snd_page_alloc(U) iTCO_vendor_support(U) joydev(U) ext4(U) mbcache(U) jbd2(U) cryptd(U) aes_x86_64(U) aes_generic(U) xts(U) gf128mul(U) dm_crypt(U) dm_multipath(U) sd_mod(U) crc_t10dif(U) yenta_socket(U) rsrc_nonstatic(U) video(U) output(U) firewire_ohci(U) firewire_core(U) crc_itu_t(U) ahci(U) radeon(U) ttm(U) drm_kms_helper(U) drm(U) i2c_algo_bit(U) i2c_core(U) dm_mod(U) [last unloaded: scsi_wait_scan]
Pid: 3704, comm: ksmtuned Not tainted 2.6.32-19.el6.x86_64 #1 2768Z28
RIP: 0010:[<ffffffff8114938c>]  [<ffffffff8114938c>] cache_alloc_refill+0x1ec/0x240
RSP: 0018:ffff8800bc4e5cc0  EFLAGS: 00010046
RAX: 0000000000000032 RBX: ffff88013bf10ac0 RCX: ffff880102c18f00
RDX: ffff88013be7d9c0 RSI: 0000000000000090 RDI: ffff880102c18000
RBP: ffff8800bc4e5d10 R08: ffff88013be7d9c0 R09: 0000000000000000
R10: 0000000000013510 R11: ffff88011fe3ffac R12: ffff88013bec7400
R13: ffff88013be7d9c0 R14: 0000000000000032 R15: ffff880102c18000
FS:  00007f7d100e2700(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000043d9d0 CR3: 00000000aad03000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process ksmtuned (pid: 3704, threadinfo ffff8800bc4e4000, task ffff88010887b4c0)
Stack:
 00007f7d00000032 000412d000000000 ffff88013be7d9e0 ffff88013be7d9d0
<0> ffff8800bc4e5d20 ffff880076454558 00000000000000d0 ffff88013bf10ac0
<0> 00000000000000d0 0000000000000246 ffff8800bc4e5d50 ffffffff8114a288
Call Trace:
 [<ffffffff8114a288>] kmem_cache_alloc+0x1c8/0x220
 [<ffffffff81132b61>] anon_vma_fork+0x51/0xa0
 [<ffffffff81065592>] dup_mm+0x232/0x4d0
 [<ffffffff810665d1>] copy_process+0xd31/0x1380
 [<ffffffff81066de4>] do_fork+0x94/0x480
 [<ffffffff8112a6be>] ? handle_mm_fault+0x1ee/0x2b0
 [<ffffffff8115b4ed>] ? fd_install+0x3d/0x70
 [<ffffffff810cb172>] ? audit_syscall_entry+0x242/0x270
 [<ffffffff81011548>] sys_clone+0x28/0x30
 [<ffffffff81013493>] stub_clone+0x13/0x20
 [<ffffffff81013172>] ? system_call_fastpath+0x16/0x1b
Code: 5c 41 5d 41 5e 41 5f c9 c3 66 0f 1f 44 00 00 49 8b 55 10 48 8b 75 c8 4c 89 ff e8 90 ba 10 00 8b 45 b0 e9 79 ff ff ff 0f 0b eb fe <0f> 0b 66 90 eb fc 8b 55 b8 8b 75 bc 31 c9 48 89 df e8 ce f8 ff 
RIP  [<ffffffff8114938c>] cache_alloc_refill+0x1ec/0x240
 RSP <ffff8800bc4e5cc0>
---[ end trace 308aa13a7ee51d8a ]---
Kernel panic - not syncing: Fatal exception
Pid: 3704, comm: ksmtuned Tainted: G      D    2.6.32-19.el6.x86_64 #1
Call Trace:
 [<ffffffff814bfd69>] panic+0x78/0x137
 [<ffffffff814c3d1c>] oops_end+0xdc/0xf0
 [<ffffffff8101723b>] die+0x5b/0x90
 [<ffffffff814c35c4>] do_trap+0xc4/0x160

Comment 3 Pekka Järveläinen 2010-05-24 07:11:44 UTC
after successful start:
top - 10:01:22 up 10 min,  6 users,  load average: 0.38, 0.42, 0.25
Tasks: 207 total,   2 running, 205 sleeping,   0 stopped,   0 zombie
Cpu(s):  8.6%us,  3.0%sy,  0.0%ni, 84.4%id,  3.9%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   3987008k total,  2399288k used,  1587720k free,    52128k buffers
Swap:  6094840k total,        0k used,  6094840k free,   679016k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND 
 3588 pj        20   0  549m  61m  21m D 10.3  1.6   0:01.39 firefox             
 3432 pj        40   0 2562m 655m  18m S  6.6 16.8   1:21.46 java               
 3637 pj        40   0  950m  82m  16m S  6.6  2.1   0:02.69 chrome             
 3052 pj        20   0 1412m  40m  23m R  5.6  1.1   0:07.42 chrome             
 2408 root      40   0  813m 128m  16m S  2.0  3.3   0:17.15 Xorg               
 2988 pj        40   0  289m 7920 6376 S  2.0  0.2   0:07.97 multiload-apple    
 3732 pj        40   0 14912 1248  884 R  0.7  0.0   0:00.08 top                
  528 root      20   0     0    0    0 S  0.3  0.0   0:13.56 kcryptd            

I have also eclipse running. The machine is lenovo thinkpad t400.

Comment 5 James M. Leddy 2010-05-24 14:46:09 UTC
Hmm, looks like the bug is in ksm. If you want a quick workaround, try disabling ksm.

Comment 6 RHEL Program Management 2010-06-07 16:06:52 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 7 Quentin Barnes 2010-06-16 22:20:13 UTC
I think I may be seeing the same panic.

This is the second time my box has paniced with RHEL6 Beta.  I
didn't capture the earlier panic because kexec/kdump isn't working
(Bug #596223 maybe?) on my box, so I set up another machine to
capture serial console output from my RHEL6B box (which had the side
effect of truncating the lines at 80 columns) and patiently waited
for it to happen again.  So far, panics about once a week.  With
both panics, I was doing the same thing, using mutt to respond to
emails, so both panics could be the same.  After all, we know how
demanding on system resources that "mutt" app can be!  :-)


 ------------[ cut here ]------------
kernel BUG at mm/slab.c:2974!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/kernel/mm/ksm/run
CPU 0
Modules linked in: fuse(U) ipt_MASQUERADE(U) iptable_nat(U) nf_nat(U) bridge(U)]
Pid: 21503, comm: mutt Not tainted 2.6.32-19.el6.x86_64 #1 HP Compaq dc5700 Micr
RIP: 0010:[<ffffffff8114938c>]  [<ffffffff8114938c>] cache_alloc_refill+0x1ec/00
RSP: 0018:ffff88007b3b3cc0  EFLAGS: 00010046
RAX: 000000000000002f RBX: ffff88007f310ac0 RCX: ffff88004eadcc30
RDX: ffff880008b0c000 RSI: 0000000000000090 RDI: ffff88004eadc000
RBP: ffff88007b3b3d10 R08: ffff880008b0c000 R09: 0000000000000000
R10: 00000000006c8000 R11: 0000000000000000 R12: ffff88007f2d7800
R13: ffff88007f27bac0 R14: 000000000000002f R15: ffff88004eadc000
FS:  00007fdd081737c0(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fdd08847000 CR3: 0000000079456000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Process mutt (pid: 21503, threadinfo ffff88007b3b2000, task ffff88004f6ea080)
Stack:
 000000000000002f 000412d000000000 ffff88007f27bae0 ffff88007f27bad0
<0> ffff88007b3b3d20 ffff88004e8a3138 00000000000000d0 ffff88007f310ac0
<0> 00000000000000d0 0000000000000246 ffff88007b3b3d50 ffffffff8114a288
Call Trace:
 [<ffffffff8114a288>] kmem_cache_alloc+0x1c8/0x220
 [<ffffffff81132b61>] anon_vma_fork+0x51/0xa0
 [<ffffffff81065592>] dup_mm+0x232/0x4d0
 [<ffffffff810665d1>] copy_process+0xd31/0x1380
 [<ffffffff81094102>] ? creds_are_invalid+0x32/0x60
 [<ffffffff81066de4>] do_fork+0x94/0x480
 [<ffffffff8107dc5f>] ? do_sigaction+0x18f/0x1c0
 [<ffffffff810cb172>] ? audit_syscall_entry+0x242/0x270
 [<ffffffff81011548>] sys_clone+0x28/0x30
 [<ffffffff81013493>] stub_clone+0x13/0x20
 [<ffffffff81013172>] ? system_call_fastpath+0x16/0x1b
Code: 5c 41 5d 41 5e 41 5f c9 c3 66 0f 1f 44 00 00 49 8b 55 10 48 8b 75 c8 4c 8
RIP  [<ffffffff8114938c>] cache_alloc_refill+0x1ec/0x240
 RSP <ffff88007b3b3cc0>
---[ end trace 81822375f7261b56 ]---
Kernel panic - not syncing: Fatal exception
Pid: 21503, comm: mutt Tainted: G      D    2.6.32-19.el6.x86_64 #1
Call Trace:
 [<ffffffff814bfd69>] panic+0x78/0x137
 [<ffffffff814c3d1c>] oops_end+0xdc/0xf0
 [<ffffffff8101723b>] die+0x5b/0x90
 [<ffffffff814c35c4>] do_trap+0xc4/0x160
 [<ffffffff81014cb5>] do_invalid_op+0x95/0xb0
 [<ffffffff8114938c>] ? cache_alloc_refill+0x1ec/0x240
 [<ffffffff81125f0a>] ? copy_pte_range+0x2da/0x490
 [<ffffffff81013f5b>] invalid_op+0x1b/0x20
 [<ffffffff8114938c>] ? cache_alloc_refill+0x1ec/0x240
 [<ffffffff8114a288>] kmem_cache_alloc+0x1c8/0x220
 [<ffffffff81132b61>] anon_vma_fork+0x51/0xa0
 [<ffffffff81065592>] dup_mm+0x232/0x4d0
 [<ffffffff810665d1>] copy_process+0xd31/0x1380
 [<ffffffff81094102>] ? creds_are_invalid+0x32/0x60
 [<ffffffff81066de4>] do_fork+0x94/0x480
 [<ffffffff8107dc5f>] ? do_sigaction+0x18f/0x1c0
 [<ffffffff810cb172>] ? audit_syscall_entry+0x242/0x270
 [<ffffffff81011548>] sys_clone+0x28/0x30
 [<ffffffff81013493>] stub_clone+0x13/0x20
 [<ffffffff81013172>] ? system_call_fastpath+0x16/0x1b
[drm:drm_fb_helper_panic] *ERROR* panic occurred, switching back to text console

Comment 8 Quentin Barnes 2010-06-17 00:53:07 UTC
I just got hit with it again within a couple of hours.  So much for the "a week between panics".  This time "ksmtuned" was fingered as the culprit.

I would assume the remark in Comment #5 means writing 0 or 2 to "/sys/kernel/mm/ksm/run"?

Comment 9 Pekka Järveläinen 2010-06-17 04:41:00 UTC
 And remember also
/sbin/chkconfig --level 345 ksm* off

Comment 10 RHEL Program Management 2010-07-15 14:38:53 UTC
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 11 Quentin Barnes 2010-07-15 14:43:38 UTC
How can frequent kernel panics when using the system normally not be considered a "blocker" issue?

Comment 12 Qian Cai 2010-07-20 00:55:46 UTC

*** This bug has been marked as a duplicate of bug 585926 ***


Note You need to log in before you can comment on or make changes to this bug.