So I was just innocently doing some F19 Final TC5 testing, booting a VM, and my system suddenly appeared to freeze. I could ssh in from my laptop; doing journalctl -a showed a whole bunch of kernel traces, with something about 'recursive error fixed, but reboot is required!' on the end. So like an idiot I rebooted, and now the kernel traces aren't in journalctl -a any more. But a bunch of kernel errors have shown up in abrt, which is refusing to report them because the kernel was tainted 'GW'. So far as I can tell all that means is 'Taint on warning', which...I don't know what that means. But there's clearly a bug here, so I'm reporting it manually. Here are the backtraces abrt has: WARNING: at lib/list_debug.c:36 __list_add+0x8a/0xc0() Hardware name: System Product Name list_add double add: new=ffff88041e8725d8, prev=ffff880425cff928, next=ffff88041e8725d8. Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Tainted: G W 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306b00>] ? btree_merge+0x100/0x100 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306b8a>] __list_add+0x8a/0xc0 [<ffffffffa00a2e55>] ttm_dma_populate+0x375/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b WARNING: at lib/list_debug.c:62 __list_del_entry+0x82/0xd0() Hardware name: System Product Name list_del corruption. next->prev should be ffff88041e8725d8, but was ffff880425cff928 Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Tainted: G W 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306c00>] ? __list_del_entry+0x40/0xd0 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306c42>] __list_del_entry+0x82/0xd0 [<ffffffffa00a2e42>] ttm_dma_populate+0x362/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b WARNING: at lib/list_debug.c:36 __list_add+0x8a/0xc0() Hardware name: System Product Name list_add double add: new=ffff88041e8725d8, prev=ffff880425cff928, next=ffff88041e8725d8. Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Tainted: G W 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306b00>] ? btree_merge+0x100/0x100 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306b8a>] __list_add+0x8a/0xc0 [<ffffffffa00a2e55>] ttm_dma_populate+0x375/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b WARNING: at lib/list_debug.c:62 __list_del_entry+0x82/0xd0() Hardware name: System Product Name list_del corruption. next->prev should be ffff88041e8725d8, but was ffff880425cff928 Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Tainted: G W 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306c00>] ? __list_del_entry+0x40/0xd0 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306c42>] __list_del_entry+0x82/0xd0 [<ffffffffa00a2e42>] ttm_dma_populate+0x362/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b WARNING: at lib/list_debug.c:33 __list_add+0xac/0xc0() Hardware name: System Product Name list_add corruption. prev->next should be next (ffff88041e8725d8), but was ffff8803ccef1700. (prev=ffff880425cff928). Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Not tainted 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306b00>] ? btree_merge+0x100/0x100 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306bac>] __list_add+0xac/0xc0 [<ffffffffa00a2e55>] ttm_dma_populate+0x375/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b BUG: unable to handle kernel paging request at 000000389f806000 IP: [<ffffffff81164382>] anon_vma_chain_link+0x12/0x40 PGD 3db87a067 PUD 3e5ea8067 PMD 412dc0067 PTE 8000000422e9c025 Oops: 0003 [#2] SMP Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi CPU 3 Pid: 6886, comm: abrt-action-ana Tainted: G D W 3.9.6-301.fc19.x86_64 #1 System manufacturer System Product Name/P8P67 DELUXE RIP: 0010:[<ffffffff81164382>] [<ffffffff81164382>] anon_vma_chain_link+0x12/0x40 RSP: 0000:ffff8803db983c80 EFLAGS: 00010246 RAX: 0000000000003535 RBX: 000000389f806000 RCX: ffff8803db983fd8 RDX: ffff8803e5603300 RSI: 000000389f806000 RDI: ffff8803efa7c508 RBP: ffff8803db983c90 R08: 0000000000017370 R09: ffffffff81165e27 R10: 0000000000000000 R11: 000000000000000e R12: ffff8803e5603300 R13: 0000000000000000 R14: 0000000000000000 R15: ffff880426b36c80 FS: 0000000000000000(0000) GS:ffff88043f4c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000389f806000 CR3: 00000003ccc81000 CR4: 00000000000427e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process abrt-action-ana (pid: 6886, threadinfo ffff8803db982000, task ffff880426485dc0) Stack: ffff8803efa7c508 ffff8803e5603300 ffff8803db983cc8 ffffffff81165de2 ffff8803efa7c508 0000000000000029 0000000000000017 ffff88038aded7f8 0000000000000000 ffff8803db983d48 ffffffff81156600 ffff88043f7ddb00 Call Trace: [<ffffffff81165de2>] anon_vma_prepare+0xa2/0x140 [<ffffffff81156600>] __do_fault+0x310/0x570 [<ffffffff8129446c>] ? cred_has_capability+0x5c/0x100 [<ffffffff81159713>] handle_pte_fault+0x93/0xa40 [<ffffffff810498d2>] ? pte_alloc_one+0x32/0x40 [<ffffffff811572a8>] ? __pte_alloc+0x88/0x140 [<ffffffff8115aed1>] handle_mm_fault+0x291/0x660 [<ffffffff811614a4>] ? mmap_region+0x1f4/0x5a0 [<ffffffff8164a196>] __do_page_fault+0x146/0x510 [<ffffffff81161b55>] ? do_mmap_pgoff+0x305/0x3c0 [<ffffffff812948f3>] ? file_map_prot_check+0x63/0xd0 [<ffffffff8114c549>] ? vm_mmap_pgoff+0x99/0xc0 [<ffffffff8164a56e>] do_page_fault+0xe/0x10 [<ffffffff81646c18>] page_fault+0x28/0x30 Code: d5 b8 f4 ff ff ff 45 31 e4 eb cb 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 41 54 49 89 d4 53 48 89 f3 <48> 89 3e 48 89 53 08 48 8b 57 78 48 8d 77 78 48 8d 7b 10 e8 66 RIP [<ffffffff81164382>] anon_vma_chain_link+0x12/0x40 RSP <ffff8803db983c80> CR2: 000000389f806000 general protection fault: 0000 [#3] SMP Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi CPU 3 Pid: 6886, comm: abrt-action-ana Tainted: G D W 3.9.6-301.fc19.x86_64 #1 System manufacturer System Product Name/P8P67 DELUXE RIP: 0010:[<ffffffff810c4ea0>] [<ffffffff810c4ea0>] acct_collect+0x60/0x1b0 RSP: 0000:ffff8803db983938 EFLAGS: 00010206 RAX: 00ededed00ed0000 RBX: ffff880409ce3b80 RCX: ffff8803db983fd8 RDX: 000077fc24900d20 RSI: 0000000000000001 RDI: ffff880426b36ce8 RBP: ffff8803db983958 R08: ffff880426485dc0 R09: 0000000000000000 R10: 0000000000000000 R11: ffff8803db98370e R12: ffff880426485dc0 R13: 0000000000000009 R14: 0000000000000000 R15: ffff880426485dc0 FS: 0000000000000000(0000) GS:ffff88043f4c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000389f806000 CR3: 00000003ccc81000 CR4: 00000000000427e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process abrt-action-ana (pid: 6886, threadinfo ffff8803db982000, task ffff880426485dc0) Stack: 0000000000000009 0000000000001ae6 0000000000000246 0000000000000003 ffff8803db9839e0 ffffffff81062b00 00000001db9839c0 ffffffff8163c312 0000000000000010 ffff8803db9839d0 ffff8803db983990 0000000000000000 Call Trace: [<ffffffff81062b00>] do_exit+0x7e0/0xa20 [<ffffffff8163c312>] ? printk+0x67/0x69 [<ffffffff816477a3>] oops_end+0xa3/0xe0 [<ffffffff8163bc48>] no_context+0x263/0x270 [<ffffffff8163bcc8>] __bad_area_nosemaphore+0x73/0x1cc [<ffffffff811381d3>] ? get_page_from_freelist+0x3b3/0x930 [<ffffffff8163c045>] bad_area+0x43/0x4a [<ffffffff8164a4ac>] __do_page_fault+0x45c/0x510 [<ffffffff8118d4ae>] ? lookup_page_cgroup_used+0xe/0x30 [<ffffffff81194761>] ? mem_cgroup_bad_page_check+0x21/0x30 [<ffffffff811383d3>] ? get_page_from_freelist+0x5b3/0x930 [<ffffffff811388b6>] ? __alloc_pages_nodemask+0x166/0xa30 [<ffffffff8164a56e>] do_page_fault+0xe/0x10 [<ffffffff81646c18>] page_fault+0x28/0x30 [<ffffffff81165e27>] ? anon_vma_prepare+0xe7/0x140 [<ffffffff81164382>] ? anon_vma_chain_link+0x12/0x40 [<ffffffff81165de2>] anon_vma_prepare+0xa2/0x140 [<ffffffff81156600>] __do_fault+0x310/0x570 [<ffffffff8129446c>] ? cred_has_capability+0x5c/0x100 [<ffffffff81159713>] handle_pte_fault+0x93/0xa40 [<ffffffff810498d2>] ? pte_alloc_one+0x32/0x40 [<ffffffff811572a8>] ? __pte_alloc+0x88/0x140 [<ffffffff8115aed1>] handle_mm_fault+0x291/0x660 [<ffffffff811614a4>] ? mmap_region+0x1f4/0x5a0 [<ffffffff8164a196>] __do_page_fault+0x146/0x510 [<ffffffff81161b55>] ? do_mmap_pgoff+0x305/0x3c0 [<ffffffff812948f3>] ? file_map_prot_check+0x63/0xd0 [<ffffffff8114c549>] ? vm_mmap_pgoff+0x99/0xc0 [<ffffffff8164a56e>] do_page_fault+0xe/0x10 [<ffffffff81646c18>] page_fault+0x28/0x30 Code: 00 00 00 74 56 49 8b 84 24 90 02 00 00 48 8d 78 68 e8 35 ef 57 00 49 8b 84 24 90 02 00 00 48 8b 00 48 85 c0 74 1c 31 d2 0f 1f 00 <48> 03 50 08 48 2b 10 48 8b 40 10 48 85 c0 75 f0 48 c1 ea 0a 49 RIP [<ffffffff810c4ea0>] acct_collect+0x60/0x1b0 RSP <ffff8803db983938> The first five look like a loop, then the last two are different. The last (seventh) one adds the taint flag 'D', which is 'kernel has oopsed before' - obviously the sixth one.
The "W" flag means the kernel has had a WARN or WARN_ON condition trigger before. It adds the flag so that you know something has already gone wrong before the oops it's currently printing out. In general, you try and find the first oops and work from there. Assuming those are all from the same boot (do you have timestamps?), this is the first error: WARNING: at lib/list_debug.c:33 __list_add+0xac/0xc0() Hardware name: System Product Name list_add corruption. prev->next should be next (ffff88041e8725d8), but was ffff8803ccef1700. (prev=ffff880425cff928). Modules linked in: nfsv4 nfs fuse ebtable_nat xt_CHECKSUM bridge stp llc arc4 md4 nls_utf8 cifs dns_resolver fscache vhost_net tun macvtap macvlan nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables rfcomm bnep snd_hda_codec_realtek iTCO_wdt iTCO_vendor_support mperf coretemp kvm_intel eeepc_wmi asus_wmi sparse_keymap kvm crc32_pclmul crc32c_intel ghash_clmulni_intel uvcvideo joydev snd_usb_audio snd_hda_intel videobuf2_vmalloc snd_hda_codec videobuf2_memops snd_usbmidi_lib videobuf2_core btusb videodev snd_rawmidi microcode snd_hwdep bluetooth media snd_seq usb_storage i2c_i801 serio_raw snd_seq_device shpchp snd_pcm rfkill lpc_ich mfd_core e1000e snd_page_alloc snd_timer snd mei ptp soundcore pps_core nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc uinput nouveau video i2c_algo_bit drm_kms_helper mxm_wmi ttm firewire_ohci drm firewire_core crc_itu_t i2c_core wmi Pid: 617, comm: Xorg Not tainted 3.9.6-301.fc19.x86_64 #1 Call Trace: [<ffffffff81306b00>] ? btree_merge+0x100/0x100 [<ffffffff8105cc26>] warn_slowpath_common+0x66/0x80 [<ffffffff8105cc8c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff81306bac>] __list_add+0xac/0xc0 [<ffffffffa00a2e55>] ttm_dma_populate+0x375/0x9e0 [ttm] [<ffffffff811819c0>] ? __kmalloc+0x230/0x250 [<ffffffff812aa06c>] ? mls_level_isvalid+0x4c/0x150 [<ffffffffa012f6df>] nouveau_ttm_tt_populate+0x1ef/0x260 [nouveau] [<ffffffffa0099c8e>] ttm_tt_bind+0x3e/0x70 [ttm] [<ffffffffa009bfcf>] ttm_bo_handle_move_mem+0x54f/0x5e0 [ttm] [<ffffffffa012e2cc>] ? nouveau_gart_manager_new+0x3c/0x80 [nouveau] [<ffffffffa009c691>] ? ttm_bo_mem_space+0x171/0x350 [ttm] [<ffffffffa009ce27>] ttm_bo_move_buffer+0x117/0x140 [ttm] [<ffffffffa009ceea>] ttm_bo_validate+0x9a/0x110 [ttm] [<ffffffffa009d25a>] ttm_bo_init+0x2fa/0x3b0 [ttm] [<ffffffffa013010d>] nouveau_bo_new+0x1dd/0x2c0 [nouveau] [<ffffffffa012ed20>] ? nv10_bo_put_tile_region+0x60/0x60 [nouveau] [<ffffffffa013261d>] nouveau_gem_new+0x5d/0x130 [nouveau] [<ffffffffa0132795>] nouveau_gem_ioctl_new+0xa5/0x1b0 [nouveau] [<ffffffffa003b2c9>] drm_ioctl+0x4e9/0x5b0 [drm] [<ffffffffa01326f0>] ? nouveau_gem_new+0x130/0x130 [nouveau] [<ffffffff811ab295>] do_vfs_ioctl+0x305/0x520 [<ffffffff8129475e>] ? file_has_perm+0x8e/0xa0 [<ffffffff811ab531>] sys_ioctl+0x81/0xa0 [<ffffffff8164ea99>] system_call_fastpath+0x16/0x1b
Yes, they are all from the same boot, and they're the only kernel traces abrt lists for that boot. I listed them in reverse order as presented by the abrt gui - it usually lists traces from first to last, so you reverse the order to get them in chronological order. the 'time' file in the abrt reports, which I think is a Unix time timestamp, is 1371621947 for the first five traces - that's Wed, 19 Jun 2013 06:05:47 GMT - and 1371622259 for the last two, which is Wed, 19 Jun 2013 06:10:59 GMT . Indeed the 'fifth' one shows not as tainted but 'incomplete' according to abrt; I'm not sure why it thinks it's incomplete, any idea? Well, I'll ask the abrt folks.
Ben, this is on the good old GeForce 9600 GT, 10de:0622 .
I'm going to bet this is the random-ish corruption from 976789 and isn't a graphics problem at all... One the kmalloc() lists get messed up, everything goes wrong...
*** This bug has been marked as a duplicate of bug 976789 ***
as long as you fix it, I don't care what you call it ;)
Until you get on the test kernel from the bug, i'd suggest not using virtio for the nic in your VMs.