Description of problem: Completing an nfs KVM install of rawhide using virtio disks (on an F-11 host) panics while rebooting Version-Release number of selected component (if applicable): * Host (x86_64): * kernel-2.6.29.6-217.2.7.fc11.x86_64 * libvirt-0.6.2-14.fc11.x86_64 * qemu-system-x86-0.10.5-3.fc11.x86_64 * Guest (i386): * kernel-2.6.31-0.125.4.2.rc5.git2.fc12.i686 How reproducible: * Only saw this 1 time so far Steps to Reproduce: 1. Perform an automated kickstart install over nfs on a KVM virtio guest Actual results: disabling swap... /dev/mapper/vg_test1197-lv_swap unmounting filesystems... /mnt/runtime done disabling /dev/loop0 LOOP_CLR_FD failed: 16 /proc done /dev/pts done /sys done /mnt/stage2 done /selinux done /mnt/sysimage/boot done /mnt/sysimage/dev/pts done /mnt/sysimage/dev/shm done /mnt/sysimage/dev done /mnt/sysimage/proc done /mnt/sysimage/sys done /mnt/sysimage/selinux done /mnt/sysimage done BUG: unable to handle kernel paging request at fffb680c IP: [<c07740ca>] skb_copy_from_linear_data_offset+0x2c/0x4e *pde = 00009067 *pte = 37842163 Oops: 0002 [#1] SMP last sysfs file: /sys/devices/virtio-pci/virtio1/block/vda/removable Modules linked in: aes_generic fcoe libfcoe libfc scsi_transport_fc scsi_tgt ipv6 xts lrw gf128mul sha256_generic cbc dm_crypt dm_round_robin dm_multipath linear raid10 raid456 raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 nfs lockd fscache nfs_acl auth_rpcgss sunrpc virtio_net virtio_blk virtio_pci iscsi_ibft iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi squashfs pcspkr edd floppy cramfs Pid: 192, comm: wpa_supplicant Not tainted (2.6.31-0.125.4.2.rc5.git2.fc12.i686 #1) EIP: 0060:[<c07740ca>] EFLAGS: 00210282 CPU: 0 EIP is at skb_copy_from_linear_data_offset+0x2c/0x4e EAX: f5ea8f00 EBX: 000005a8 ECX: 0000016a EDX: c213f304 ESI: c213f304 EDI: fffb680c EBP: c2161ab8 ESP: c2161aa8 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 Process wpa_supplicant (pid: 192, ti=c2160000 task=f63cab80 task.ti=c2160000) Stack: 481d84b0 000005a8 000005a8 000005a8 c2161aec c0774a72 000005a8 00000001 <0> c2161ad8 f5ea8f00 fffb680c 00000000 000005a8 481d84b0 c2161b68 000005a8 <0> 00000000 c2161b04 f7d7af3c 000005a8 481d84b0 0000080c f62aa748 c2161b34 Call Trace: [<c0774a72>] ? skb_copy_bits+0x5e/0x1a0 [<f7d7af3c>] ? xdr_skb_read_bits+0x34/0x60 [sunrpc] [<f7d7ad89>] ? xdr_partial_copy_from_skb+0x121/0x185 [sunrpc] [<f7d7af08>] ? xdr_skb_read_bits+0x0/0x60 [sunrpc] [<f7d7c700>] ? xs_tcp_data_recv+0x371/0x53b [sunrpc] [<f7d7af08>] ? xdr_skb_read_bits+0x0/0x60 [sunrpc] [<c07b6dbc>] ? tcp_read_sock+0x7d/0x193 [<f7d7c38f>] ? xs_tcp_data_recv+0x0/0x53b [sunrpc] [<f7d7c36b>] ? xs_tcp_data_ready+0x6a/0x8e [sunrpc] [<c07be7db>] ? tcp_rcv_established+0x4fe/0x63a [<c07c5603>] ? tcp_v4_do_rcv+0x171/0x2c7 [<c07c695c>] ? tcp_v4_rcv+0x3ea/0x5e2 [<c07ab5b4>] ? ip_local_deliver_finish+0x13f/0x1ee [<c07ab6d7>] ? ip_local_deliver+0x74/0x8d [<c07ab05e>] ? ip_rcv_finish+0x31f/0x35a [<c07ab2bb>] ? ip_rcv+0x222/0x266 [<c077df66>] ? netif_receive_skb+0x38e/0x3bf [<f7d39714>] ? virtnet_poll+0x4ca/0x633 [virtio_net] [<c077b9e0>] ? __rcu_read_lock+0x0/0x45 [<c077e724>] ? net_rx_action+0xa7/0x1d3 [<c044adbb>] ? __do_softirq+0x60/0x192 [<c044ae23>] ? __do_softirq+0xc8/0x192 [<c044af36>] ? do_softirq+0x49/0x7f [<c044b08a>] ? irq_exit+0x48/0x8c [<c0405705>] ? do_IRQ+0x92/0xb7 [<c0404095>] ? common_interrupt+0x35/0x3c [<c04264c5>] ? paravirt_leave_lazy_mmu+0x0/0x22 [<c042593d>] ? kvm_leave_lazy_mmu+0x5c/0x7e [<c04d74a5>] ? unmap_vmas+0x489/0x5c0 [<c0446200>] ? task_unlock+0xe/0x34 [<c04dc13f>] ? exit_mmap+0xb2/0x113 [<c04421b3>] ? mmput+0x57/0xc0 [<c0446492>] ? exit_mm+0xeb/0x104 [<c0447fd0>] ? do_exit+0x19e/0x648 [<c046f278>] ? trace_hardirqs_on_caller+0x122/0x155 [<c04484ec>] ? do_group_exit+0x72/0x99 [<c044853a>] ? sys_exit_group+0x27/0x3c [<c0403a50>] ? syscall_call+0x7/0xb Code: 89 e5 57 56 53 83 ec 04 0f 1f 44 00 00 8b 5d 08 89 cf 65 8b 0d 14 00 00 00 89 4d f0 31 c9 89 d9 c1 e9 02 03 90 b4 00 00 00 89 d6 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 8b 45 f0 65 33 05 14 00 00 00 EIP: [<c07740ca>] skb_copy_from_linear_data_offset+0x2c/0x4e SS:ESP 0068:c2161aa8 CR2: 00000000fffb680c ---[ end trace f6d9dc8fdbaaf440 ]--- Kernel panic - not syncing: Fatal exception in interrupt Pid: 192, comm: wpa_supplicant Tainted: G D 2.6.31-0.125.4.2.rc5.git2.fc12.i686 #1 Call Trace: [<c081ab29>] ? printk+0x22/0x39 [<c081aa60>] panic+0x4b/0xf2 [<c081e964>] oops_end+0xa0/0xc0 [<c0427687>] no_context+0x11a/0x135 [<c04277e4>] __bad_area_nosemaphore+0x142/0x15b [<c04268f4>] ? pvclock_clocksource_read+0x59/0xf4 [<c04268f4>] ? pvclock_clocksource_read+0x59/0xf4 [<c042781f>] bad_area_nosemaphore+0x22/0x39 [<c081fc5c>] do_page_fault+0x168/0x2fa [<c081faf4>] ? do_page_fault+0x0/0x2fa [<c081dec8>] error_code+0x78/0x80 [<c07740ca>] ? skb_copy_from_linear_data_offset+0x2c/0x4e [<c0774a72>] skb_copy_bits+0x5e/0x1a0 [<f7d7af3c>] xdr_skb_read_bits+0x34/0x60 [sunrpc] [<f7d7ad89>] xdr_partial_copy_from_skb+0x121/0x185 [sunrpc] [<f7d7af08>] ? xdr_skb_read_bits+0x0/0x60 [sunrpc] [<f7d7c700>] xs_tcp_data_recv+0x371/0x53b [sunrpc] [<f7d7af08>] ? xdr_skb_read_bits+0x0/0x60 [sunrpc] [<c07b6dbc>] tcp_read_sock+0x7d/0x193 [<f7d7c38f>] ? xs_tcp_data_recv+0x0/0x53b [sunrpc] [<f7d7c36b>] xs_tcp_data_ready+0x6a/0x8e [sunrpc] [<c07be7db>] tcp_rcv_established+0x4fe/0x63a [<c07c5603>] tcp_v4_do_rcv+0x171/0x2c7 [<c07c695c>] tcp_v4_rcv+0x3ea/0x5e2 [<c07ab5b4>] ip_local_deliver_finish+0x13f/0x1ee [<c07ab6d7>] ip_local_deliver+0x74/0x8d [<c07ab05e>] ip_rcv_finish+0x31f/0x35a [<c07ab2bb>] ip_rcv+0x222/0x266 [<c077df66>] netif_receive_skb+0x38e/0x3bf [<f7d39714>] virtnet_poll+0x4ca/0x633 [virtio_net] [<c077b9e0>] ? __rcu_read_lock+0x0/0x45 [<c077e724>] net_rx_action+0xa7/0x1d3 [<c044adbb>] ? __do_softirq+0x60/0x192 [<c044ae23>] __do_softirq+0xc8/0x192 [<c044af36>] do_softirq+0x49/0x7f [<c044b08a>] irq_exit+0x48/0x8c [<c0405705>] do_IRQ+0x92/0xb7 [<c0404095>] common_interrupt+0x35/0x3c [<c04264c5>] ? paravirt_leave_lazy_mmu+0x0/0x22 [<c042593d>] ? kvm_leave_lazy_mmu+0x5c/0x7e [<c04d74a5>] unmap_vmas+0x489/0x5c0 [<c0446200>] ? task_unlock+0xe/0x34 [<c04dc13f>] exit_mmap+0xb2/0x113 [<c04421b3>] mmput+0x57/0xc0 [<c0446492>] exit_mm+0xeb/0x104 [<c0447fd0>] do_exit+0x19e/0x648 [<c046f278>] ? trace_hardirqs_on_caller+0x122/0x155 [<c04484ec>] do_group_exit+0x72/0x99 [<c044853a>] sys_exit_group+0x27/0x3c [<c0403a50>] syscall_call+0x7/0xb Expected results: Additional info:
Oh dear, this is new. It's nothing to do with the other NFS issues we're having in libvirt Looks identical to http://www.kerneloops.org/raw.php?rawid=642150 Pity it's not reproducible Could be a pvmmu issue perhaps?
The bogus pointer (fffb680c) comes from EDI, which is supplied by the sunrpc code. So it doesn't appear to be a virtio-net or even general networking problem. As the only report appears with KVM, I agree with you that perhaps it's a KVM MM issue.
Justin has just committed this patch from Marcelo http://patchwork.kernel.org/patch/43695/ * Thu Sep 03 2009 Justin M.Forbes <jforbes> - Fix xen guest with stack protector. (#508120) - Small kvm fixes.