Additional info: BUG: soft lockup - CPU#6 stuck for 22s! [qemu-kvm:872] Modules linked in: bluetooth cpufreq_stats fuse ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat xt_CHECKSUM iptable_mangle bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ip6table_filter ip6_tables be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i cxgb3 mdio libcxgbi ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi snd_hda_codec_hdmi snd_hda_codec_realtek uvcvideo videobuf2_vmalloc videobuf2_memops snd_hda_intel snd_hda_codec arc4 iwldvm videobuf2_core mac80211 videodev media snd_hwdep snd_seq coretemp iwlwifi snd_seq_device snd_pcm r8169 cfg80211 iTCO_wdt iTCO_vendor_support lpc_ich snd_page_alloc mii dell_wmi snd_timer mfd_core microcode dell_laptop dcdbas i2c_i801 mei sparse_keymap snd rfkill soundcore joydev vhost_net tun macvtap macvlan nfsd auth_rpcgss kvm_intel nfs_acl kvm lockd sunrpc binfmt_misc uinput i915 nouveau mxm_wmi i2c_algo_bit drm_kms_helper ttm crc32c_intel drm ghash_clmulni_intel i2c_core wmi video CPU 6 Pid: 872, comm: qemu-kvm Not tainted 3.7.2-201.fc18.x86_64 #1 Dell Inc. Dell System XPS L702X/0XN71K RIP: 0010:[<ffffffffa0312979>] [<ffffffffa0312979>] kvm_arch_vcpu_ioctl_run+0x3d9/0x1010 [kvm] RSP: 0018:ffff8801f38fdd58 EFLAGS: 00000203 RAX: 00000893136d0d0d RBX: 000000000000000d RCX: ffff88019f94bdd0 RDX: 0000000000002010 RSI: 0000000000000000 RDI: ffff88019f94bdd0 RBP: ffff8801f38fddf8 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa02b3ac3 R13: ffff8801f38fdcd8 R14: ffffffffa02afb4c R15: 0000000000000000 FS: 00007f642efe1700(0000) GS:ffff88023e780000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 00000001ad2b5000 CR4: 00000000000427e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process qemu-kvm (pid: 872, threadinfo ffff8801f38fc000, task ffff8802308d2e40) Stack: 0000000000000000 0000000000000000 ffff8801f38fdd98 ffffffffa02b08d5 ffff8801f38fddb8 ffffffffa02b273e ffff8801f38fdfd8 ffff8801f38fddf0 ffff8802308d2e40 ffff8802308d2e40 ffff8801c1585100 fffffffe7ffbfeff Call Trace: [<ffffffffa02b08d5>] ? __vmx_load_host_state+0x105/0x150 [kvm_intel] [<ffffffffa02b273e>] ? vmx_vcpu_load+0x3e/0x1a0 [kvm_intel] [<ffffffffa030dc57>] ? kvm_arch_vcpu_load+0x57/0x1b0 [kvm] [<ffffffffa02fb202>] kvm_vcpu_ioctl+0x472/0x610 [kvm] [<ffffffff81195d3e>] ? do_readv_writev+0x18e/0x1e0 [<ffffffff811a64e9>] do_vfs_ioctl+0x99/0x580 [<ffffffff811a6a61>] sys_ioctl+0x91/0xb0 [<ffffffff810dc7dc>] ? __audit_syscall_exit+0x3ec/0x450 [<ffffffff8163dad9>] system_call_fastpath+0x16/0x1b Code: 00 a8 aa 0f 85 31 08 00 00 48 8b 05 62 77 03 00 48 89 df ff 90 38 02 00 00 c7 43 2c 00 00 00 00 48 89 83 58 20 00 00 fb 66 66 90 <66> 66 90 83 83 b8 00 00 00 01 48 8b 7d a0 e8 14 3a d8 e0 48 8b
Created attachment 690915 [details] Image capture of a kernel panic that started wit CPU soft lockup It happens very often, to the point of making my KVM installation useless. uname -r 2.6.32-279.19.1.el6.x86_64 I tried upgrading the kernel to 3.5, but the issue remains. If this is not fixed soon, KVM is history.
In my case I think the issue is a Windows 8 virtual machine. The disk is IDE, and the network interface is Virtio. In 2.6.32-279.19.1.el6.x86_64, it does not even start, shows an error saying "Your CPU does not support .." some type of CPU feature, but it is incorrect, since this is a Dell R900 server. Also, while it shows this error, it eats up 2 cores 100%. Paradoxically, this virtual machines starts just fine in Kernel 3.5.0, but then we fall into this bug after some time. If somebody analyzes my Windows 8 Virtual machine, you may find the problem. The kernel and kvm seem to be stable if I only use Linux virtual machines. I also noticed that if I change my Windows disk type to Virtio, it is several times slower to boot than when using IDE.
Alves, you should file a bug against RHEL (or CentOS if you're getting it from there) for the 3.6.32 kernel issues. Lubo, are you still seeing this with 3.8.x?
Hi Josh, in last weeks - no. May it was a problem only on that kernel/sw combination (?),