Description of problem: As a bittorrent download was ramping up, the system froze for 2 minutes. (the two minute pause is self evident). The following appeared in the log: Jan 7 17:22:45 white kernel: mismatch in kmem_cache_free: expected cache df7c9140, got df7c9200 Jan 7 17:22:45 white kernel: df7c9200 is skbuff_head_cache. Jan 7 17:22:45 white kernel: df7c9140 is skbuff_fclone_cache. Jan 7 17:22:45 white kernel: Badness in cache_free_debugcheck at mm/slab.c:2315 (Not tainted) Jan 7 17:22:45 white kernel: [<c0142731>] cache_free_debugcheck+0x86/0x198 [<c028d438>] __alloc_skb+0x10e/0x117 Jan 7 17:22:45 white kernel: [<c0142f85>] kmem_cache_free+0x1f/0x4f [<c028d438>] __alloc_skb+0x10e/0x117 Jan 7 17:22:45 white kernel: [<c02b2dd2>] tcp_sendmsg+0x153/0x97f [<c0104c02>] do_IRQ+0x6e/0x77 Jan 7 17:22:45 white kernel: [<c014007b>] background_writeout+0x12/0x8e [<c02bc1b1>] tcp_write_xmit+0x50/0x2d0 Jan 7 17:22:45 white kernel: [<c02cada9>] inet_sendmsg+0x35/0x3f [<c0289767>] sock_sendmsg+0xca/0xe4 Jan 7 17:22:45 white kernel: [<c012ca1c>] autoremove_wake_function+0x0/0x2d [<c0289767>] sock_sendmsg+0xca/0xe4 Jan 7 17:22:45 white kernel: [<c01037a6>] common_interrupt+0x1a/0x20 [<c02897a7>] kernel_sendmsg+0x26/0x2c Jan 7 17:22:45 white kernel: [<c028c59d>] sock_no_sendpage+0x5f/0x72 [<c02b2c54>] tcp_sendpage+0x33/0x5e Jan 7 17:22:45 white kernel: [<c02b2c21>] tcp_sendpage+0x0/0x5e [<e04d6e55>] xs_tcp_send_request+0x1eb/0x317 [sunrpc] Jan 7 17:22:45 white kernel: [<e04d5fa8>] xprt_transmit+0xd2/0x1c3 [sunrpc] [<e04d4d03>] call_transmit+0x6d/0x9e [sunrpc] Jan 7 17:22:45 white kernel: [<e04d894f>] __rpc_execute+0x61/0x1a9 [sunrpc] [<c0129348>] worker_thread+0x167/0x1d1 Jan 7 17:22:45 white kernel: [<e04d8ab3>] rpc_async_schedule+0x0/0x5 [sunrpc] [<c02e4331>] _spin_unlock_irq+0x5/0x7Jan 7 17:22:45 white kernel: [<c01190cd>] default_wake_function+0x0/0xc [<c01291e1>] worker_thread+0x0/0x1d1 Jan 7 17:22:45 white kernel: [<c012c5cc>] kthread+0x64/0x90 [<c012c568>] kthread+0x0/0x90 Jan 7 17:22:45 white kernel: [<c010127d>] kernel_thread_helper+0x5/0xb Jan 7 17:22:45 white kernel: slab error in cache_free_debugcheck(): cache `skbuff_head_cache': double free, or memory outside object was overwritten Jan 7 17:22:45 white kernel: [<c0142770>] cache_free_debugcheck+0xc5/0x198 [<c028d438>] __alloc_skb+0x10e/0x117 Jan 7 17:22:45 white kernel: [<c0142f85>] kmem_cache_free+0x1f/0x4f [<c028d438>] __alloc_skb+0x10e/0x117 Jan 7 17:22:45 white kernel: [<c02b2dd2>] tcp_sendmsg+0x153/0x97f [<c0104c02>] do_IRQ+0x6e/0x77 Jan 7 17:22:45 white kernel: [<c014007b>] background_writeout+0x12/0x8e [<c02bc1b1>] tcp_write_xmit+0x50/0x2d0 Jan 7 17:22:45 white kernel: [<c02cada9>] inet_sendmsg+0x35/0x3f [<c0289767>] sock_sendmsg+0xca/0xe4 Jan 7 17:22:46 white kernel: [<c012ca1c>] autoremove_wake_function+0x0/0x2d [<c0289767>] sock_sendmsg+0xca/0xe4 Jan 7 17:22:46 white kernel: [<c01037a6>] common_interrupt+0x1a/0x20 [<c02897a7>] kernel_sendmsg+0x26/0x2c Jan 7 17:22:46 white kernel: [<c028c59d>] sock_no_sendpage+0x5f/0x72 [<c02b2c54>] tcp_sendpage+0x33/0x5e Jan 7 17:22:46 white kernel: [<c02b2c21>] tcp_sendpage+0x0/0x5e [<e04d6e55>] xs_tcp_send_request+0x1eb/0x317 [sunrpc] Jan 7 17:22:46 white kernel: [<e04d5fa8>] xprt_transmit+0xd2/0x1c3 [sunrpc] [<e04d4d03>] call_transmit+0x6d/0x9e [sunrpc] Jan 7 17:22:46 white kernel: [<e04d894f>] __rpc_execute+0x61/0x1a9 [sunrpc] [<c0129348>] worker_thread+0x167/0x1d1 Jan 7 17:22:46 white kernel: [<e04d8ab3>] rpc_async_schedule+0x0/0x5 [sunrpc] [<c02e4331>] _spin_unlock_irq+0x5/0x7Jan 7 17:22:46 white kernel: [<c01190cd>] default_wake_function+0x0/0xc [<c01291e1>] worker_thread+0x0/0x1d1 Jan 7 17:22:46 white kernel: [<c012c5cc>] kthread+0x64/0x90 [<c012c568>] kthread+0x0/0x90 Jan 7 17:22:46 white kernel: [<c010127d>] kernel_thread_helper+0x5/0xb Jan 7 17:22:46 white kernel: cb74f56c: redzone 1: 0x170fc2a5, redzone 2: 0x5a5a5a5a. Jan 7 17:22:46 white kernel: ------------[ cut here ]------------ Jan 7 17:22:46 white kernel: kernel BUG at mm/slab.c:2335! Jan 7 17:22:46 white kernel: invalid operand: 0000 [#1] Jan 7 17:22:46 white kernel: last sysfs file: /devices/system/cpu/cpu0/cpufreq/scaling_cur_freq Jan 7 17:22:46 white kernel: Modules linked in: loop i915 drm parport_pc lp parport autofs4 eeprom nfs lockd nfs_acl sunrpc dm_mirror dm_mod video button battery ac ipv6 ohci1394 ieee1394 uhci_hcd ehci_hcd ipw2200 ieee80211 ieee80211_crypt b44 mii sr_mod snd_intel8x0 snd_ac97_codec snd_ac97_bus snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd i2c_i801 soundcore snd_page_alloc i2c_core ext3 jbd ahci ata_piix libata sd_mod scsi_mod Jan 7 17:22:46 white kernel: CPU: 0 Jan 7 17:22:46 white kernel: EIP: 0060:[<c01427ec>] Not tainted VLI Jan 7 17:22:46 white kernel: EFLAGS: 00210006 (2.6.14-1.1796_FC5) Jan 7 17:22:46 white kernel: EIP is at cache_free_debugcheck+0x141/0x198 Jan 7 17:22:46 white kernel: eax: cb74f4e4 ebx: cb74f04c ecx: 000000a8 edx: 00000088 Jan 7 17:22:46 white kernel: esi: df7c9200 edi: cb74f56c ebp: cb74f000 esp: d7d2ec84 Jan 7 17:22:46 white kernel: ds: 007b es: 007b ss: 0068 Jan 7 17:22:46 white kernel: Process rpciod/0 (pid: 2057, threadinfo=d7d2e000 task=d87ad550) Jan 7 17:22:46 white kernel: Stack: c028d438 cb74f570 df7c9200 df6e4720 00200246 c0142f85 cb74f570 00000700 Jan 7 17:22:46 white kernel: 00000220 00000001 c028d438 cb74f198 ca0b0618 00000100 00000000 c02b2dd2 Jan 7 17:22:46 white kernel: d7d2ecdc c0104c02 00000000 00000001 d7d2ee70 00000000 00004040 000005a8 Jan 7 17:22:46 white kernel: Call Trace: Jan 7 17:22:46 white kernel: [<c028d438>] __alloc_skb+0x10e/0x117 [<c0142f85>] kmem_cache_free+0x1f/0x4f Jan 7 17:22:46 white kernel: [<c028d438>] __alloc_skb+0x10e/0x117 [<c02b2dd2>] tcp_sendmsg+0x153/0x97f Jan 7 17:22:46 white kernel: [<c0104c02>] do_IRQ+0x6e/0x77 [<c014007b>] background_writeout+0x12/0x8e Jan 7 17:22:46 white kernel: [<c02bc1b1>] tcp_write_xmit+0x50/0x2d0 [<c02cada9>] inet_sendmsg+0x35/0x3f Jan 7 17:22:46 white kernel: [<c0289767>] sock_sendmsg+0xca/0xe4 [<c012ca1c>] autoremove_wake_function+0x0/0x2d Jan 7 17:22:46 white kernel: [<c0289767>] sock_sendmsg+0xca/0xe4 [<c01037a6>] common_interrupt+0x1a/0x20 Jan 7 17:22:46 white kernel: [<c02897a7>] kernel_sendmsg+0x26/0x2c [<c028c59d>] sock_no_sendpage+0x5f/0x72 Jan 7 17:22:46 white kernel: [<c02b2c54>] tcp_sendpage+0x33/0x5e [<c02b2c21>] tcp_sendpage+0x0/0x5e Jan 7 17:22:46 white kernel: [<e04d6e55>] xs_tcp_send_request+0x1eb/0x317 [sunrpc] [<e04d5fa8>] xprt_transmit+0xd2/0x1c3 [sunrpc] Jan 7 17:22:46 white kernel: [<e04d4d03>] call_transmit+0x6d/0x9e [sunrpc] [<e04d894f>] __rpc_execute+0x61/0x1a9 [sunrpc] Jan 7 17:22:46 white kernel: [<c0129348>] worker_thread+0x167/0x1d1 [<e04d8ab3>] rpc_async_schedule+0x0/0x5 [sunrpc] Jan 7 17:22:46 white kernel: [<c02e4331>] _spin_unlock_irq+0x5/0x7 [<c01190cd>] default_wake_function+0x0/0xc Jan 7 17:22:46 white kernel: [<c01291e1>] worker_thread+0x0/0x1d1 [<c012c5cc>] kthread+0x64/0x90 Jan 7 17:22:46 white kernel: [<c012c568>] kthread+0x0/0x90 [<c010127d>] kernel_thread_helper+0x5/0xb Jan 7 17:22:46 white kernel: Code: ff 8b 14 24 89 10 8b 5d 0c 8b 4e 10 89 f8 29 d8 31 d2 f7 f1 3b 46 1c 72 08 0f 0b 1e 09 e4 0d 30 c0 0f af c1 8d 04 03 39 c7 74 08 <0f> 0b 1f 09 e4 0d 30 c0 f6 46 19 02 74 12 89 f8 03 86 98 00 00 Continuing in 113 seconds. l: Continuing in 120 seconds. Continuing in 104 seconds. Continuing in 95 seconds. Continuing in 85 seconds. o Continuing in 77 seconds. el: tinuing in 84 seconds. Continuing in 67 seconds. Continuing in 58 seconds. Continuing in 48 seconds. Continuing in 40 seconds. el: tinuing in 47 seconds. Continuing in 30 seconds. Continuing in 21 seconds. Continuing in 11 seconds. Continuing in 2 seconds. nel: tinuing in 10 seconds. Continuing in 1 seconds. Version-Release number of selected component (if applicable): 2.6.14-1.1796_FC5 How reproducible: Unsure. I'm moved on to the latest FC5 kernel. Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Created attachment 122913 [details] Contents of /var/log/messages The log entries were line-wrapped. Here it is as an attachment.
I think I may have reproduced this in 2.6.15-1.1826.2.4_FC5. With firefox open with 27 tabs. Starting bittorrent and allowing it to ramp its speed up to 700 KB/s, eventually the system locked up, this time permanently (no log entries). The torrent I was downloading was: http://media.djangoproject.com/snakesandrubies/snakesandrubies.mp4.torrent When I didn't start firefox, bittorrent downloaded the remainder of the file (70 to 80%) without pausing.
There were some silly vm snafu's in the last week or so, current builds should be ok. Can you still reproduce this ?
Hit it again about 20 minutes ago with kernel-2.6.15-1.1854_FC5. I didn't hit it with FC5-t2, but adding another large file to bittorrent did me in. FC5-t2 didn't have a rapid increase in d/l rate, but I also wasn't watching this new file.
Created attachment 123287 [details] dmesg related contents after the 2 minute pause
And again in 2.6.15-1.1858_FC5 mismatch in kmem_cache_free: expected cache df7c9080, got df7c9140 df7c9140 is skbuff_head_cache. df7c9080 is skbuff_fclone_cache. Badness in cache_free_debugcheck at mm/slab.c:2336 (Not tainted) [<c015174f>] cache_free_debugcheck+0x86/0x198 [<c0289801>] __alloc_skb+0xe0/0xea [<c0151880>] kmem_cache_free+0x1f/0x4f [<c0289801>] __alloc_skb+0xe0/0xea [<c02ae8a7>] tcp_sendmsg+0x152/0x97e [<c0151505>] check_poison_obj+0xac/0x161 [<c02c5160>] inet_sendmsg+0x35/0x3f [<c028466a>] sock_sendmsg+0xd2/0xec [<c012ac27>] autoremove_wake_function+0x0/0x2d [<c02c5160>] inet_sendmsg+0x35/0x3f [<c028466a>] sock_sendmsg+0xd2/0xec [<c0285d64>] kernel_sendmsg+0x26/0x2c I can attach the rest if needed.
should be fixed in current build