Bug 1086058
| Summary: | fail to boot L2 guest on wildcatpass Haswell host | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | FuXiangChun <xfu> |
| Component: | kernel | Assignee: | Bandan Das <bdas> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.0 | CC: | bdas, choma, hhuang, juzhang, knoel, michen, mrezanin, pbonzini, rbalakri, rhack, virt-maint |
| Target Milestone: | rc | Keywords: | TestOnly |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | kernel-3.10.0-143.el7 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-03-05 11:51:43 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1116936 | ||
| Bug Blocks: | |||
Re-tested this bug with 3.10.0-145.el7.x86_64 (host & guest) and qemu-kvm-rhev-2.1.0-2.el7.x86_64 on host intel-wildcatpass-02.khw.lab.eng.bos.redhat.com. 1. Enable nested virt on wildcatpass host # cat /sys/module/kvm_intel/parameters/nested Y # cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y # cat /sys/module/kvm_intel/parameters/ept Y # cat /sys/module/kvm_intel/parameters/enable_apicv Y 2.Boot RHEL7.1 L1 guest on host with '-cpu host' 3.Boot L2 RHEL7.0 guest on L1 with Haswell result: 1. L0 host show call trace. [ 2767.750777] WARNING: at arch/x86/kvm/vmx.c:8686 nested_vmx_vmexit+0x282/0x290 [kvm_intel]() [ 2767.750789] Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm sg bridge stp llc intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ixgbe ptp aesni_intel pps_core lrw iTCO_wdt mdio gf128mul glue_helper iTCO_vendor_support dca ablk_helper mei_me lpc_ich cryptd pcspkr i2c_i801 mfd_core mei shpchp wmi ipmi_si ipmi_msghandler acpi_pad acpi_power_meter mperf xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ahci ttm libahci drm libata i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kvm] [ 2767.750790] CPU: 26 PID: 3659 Comm: qemu-kvm Tainted: G W -------------- 3.10.0-145.el7.x86_64 #1 [ 2767.750791] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0037.D10.1408060946 08/06/2014 [ 2767.750795] 0000000000000000 00000000085cacc2 ffff88086728fc60 ffffffff815eb72b [ 2767.750797] ffff88086728fc98 ffffffff8106a6c1 ffff880386090000 ffff8802cf661000 [ 2767.750799] 0000000000000001 ffff880864eb4440 ffff880386090030 ffff88086728fca8 [ 2767.750799] Call Trace: [ 2767.750802] [<ffffffff815eb72b>] dump_stack+0x19/0x1b [ 2767.750804] [<ffffffff8106a6c1>] warn_slowpath_common+0x61/0x80 [ 2767.750806] [<ffffffff8106a7ea>] warn_slowpath_null+0x1a/0x20 [ 2767.750808] [<ffffffffa046ddb2>] nested_vmx_vmexit+0x282/0x290 [kvm_intel] [ 2767.750810] [<ffffffffa0465e48>] ? vmx_sync_pir_to_irr+0x28/0x30 [kvm_intel] [ 2767.750812] [<ffffffffa046dec0>] vmx_check_nested_events+0x100/0x120 [kvm_intel] [ 2767.750819] [<ffffffffa04e8d2f>] vcpu_enter_guest+0x42f/0xd50 [kvm] [ 2767.750821] [<ffffffffa0469d6b>] ? vmx_deliver_posted_interrupt+0x3b/0x80 [kvm_intel] [ 2767.750827] [<ffffffffa0508722>] ? __apic_accept_irq+0x162/0x2b0 [kvm] [ 2767.750833] [<ffffffffa04ee848>] kvm_arch_vcpu_ioctl_run+0x208/0x480 [kvm] [ 2767.750837] [<ffffffffa04d8e3a>] kvm_vcpu_ioctl+0x2aa/0x580 [kvm] [ 2767.750839] [<ffffffff812015d1>] ? fsnotify+0x251/0x350 [ 2767.750841] [<ffffffff811d2765>] do_vfs_ioctl+0x2e5/0x4c0 [ 2767.750843] [<ffffffff812672de>] ? file_has_perm+0xae/0xc0 [ 2767.750845] [<ffffffff811d29e1>] SyS_ioctl+0xa1/0xc0 [ 2767.750847] [<ffffffff815fbc59>] system_call_fastpath+0x16/0x1b [ 2767.750847] ---[ end trace a300483cabb59779 ]--- [ 2767.915233] WARNING: at arch/x86/kvm/vmx.c:8686 nested_vmx_vmexit+0x282/0x290 [kvm_intel]() [ 2767.915265] Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm sg bridge stp llc intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ixgbe ptp aesni_intel pps_core lrw iTCO_wdt mdio gf128mul glue_helper iTCO_vendor_support dca ablk_helper mei_me lpc_ich cryptd pcspkr i2c_i801 mfd_core mei shpchp wmi ipmi_si ipmi_msghandler acpi_pad acpi_power_meter mperf xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ahci ttm libahci drm libata i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kvm] [ 2767.915268] CPU: 26 PID: 3659 Comm: qemu-kvm Tainted: G W -------------- 3.10.0-145.el7.x86_64 #1 [ 2767.915269] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0037.D10.1408060946 08/06/2014 [ 2767.915277] 0000000000000000 00000000085cacc2 ffff88086728fc60 ffffffff815eb72b [ 2767.915282] ffff88086728fc98 ffffffff8106a6c1 ffff880386090000 ffff8802cf661000 [ 2767.915288] 0000000000000001 ffff880864eb4440 ffff880386090030 ffff88086728fca8 [ 2767.915289] Call Trace: [ 2767.915296] [<ffffffff815eb72b>] dump_stack+0x19/0x1b [ 2767.915300] [<ffffffff8106a6c1>] warn_slowpath_common+0x61/0x80 [ 2767.915304] [<ffffffff8106a7ea>] warn_slowpath_null+0x1a/0x20 [ 2767.915311] [<ffffffffa046ddb2>] nested_vmx_vmexit+0x282/0x290 [kvm_intel] [ 2767.915316] [<ffffffffa0465e48>] ? vmx_sync_pir_to_irr+0x28/0x30 [kvm_intel] [ 2767.915322] [<ffffffffa046dec0>] vmx_check_nested_events+0x100/0x120 [kvm_intel] [ 2767.915339] [<ffffffffa04e8d2f>] vcpu_enter_guest+0x42f/0xd50 [kvm] [ 2767.915345] [<ffffffffa0469d6b>] ? vmx_deliver_posted_interrupt+0x3b/0x80 [kvm_intel] [ 2767.915363] [<ffffffffa0508722>] ? __apic_accept_irq+0x162/0x2b0 [kvm] [ 2767.915379] [<ffffffffa04ee848>] kvm_arch_vcpu_ioctl_run+0x208/0x480 [kvm] [ 2767.915391] [<ffffffffa04d8e3a>] kvm_vcpu_ioctl+0x2aa/0x580 [kvm] [ 2767.915395] [<ffffffff812015d1>] ? fsnotify+0x251/0x350 [ 2767.915413] [<ffffffff811d2765>] do_vfs_ioctl+0x2e5/0x4c0 [ 2767.915415] [<ffffffff812672de>] ? file_has_perm+0xae/0xc0 [ 2767.915417] [<ffffffff811d29e1>] SyS_ioctl+0xa1/0xc0 [ 2767.915419] [<ffffffff815fbc59>] system_call_fastpath+0x16/0x1b [ 2767.915419] ---[ end trace a300483cabb5977a ]--- 2. L2 guest boot fail(black screen) As original problem has been solved. For host call trace issue, Form QE POV, It is another issue. In order to clearly trace it. QE filed another bug 1134700. BTW, If you think should DUP this two bugs. then re-assign this bug. For QE, It is not problem. =Reasign to kernel as this is not qemu-kvm issue Re-tested this issue with 3.10.0-208.el7.x86_64 and qemu-kvm-rhev-2.1.2-13.el7.x86_64 on wildcatpass Haswell host. when booting L2 guest on L1 guest. result: error message is gong as comment 0, But Host call trace(bug 1134700). according to this test result, QE think this bug can be verified. we will continue to concern bug 1134700. Hi. All. I have still troubles on: Host CPU (real machine): processor : 23 vendor_id : GenuineIntel cpu family : 6 model : 62 model name : Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz stepping : 4 microcode : 0x428 cpu MHz : 2362.582 cache size : 15360 KB physical id : 1 siblings : 12 core id : 5 cpu cores : 6 apicid : 43 initial apicid : 43 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms bogomips : 4204.45 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: Guest - this guest is host for virtual machine which I want to run: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 62 model name : Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz stepping : 4 microcode : 0x1 cpu MHz : 2099.998 cache size : 4096 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm xsaveopt fsgsbase tsc_adjust smep erms bogomips : 4199.99 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: /usr/libexec/qemu-kvm -M pc -cpu Haswell -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -usb -device usb-tablet,id=input0 -name nested -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -vnc :3 -monitor stdio -serial unix:/tmp/ttyS0,server,nowait QEMU 1.5.3 monitor - type 'help' for more information (qemu) KVM: entry failed, hardware error 0x0 EAX=00000000 EBX=00000000 ECX=00000000 EDX=000306c1 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000e05b EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 000f0000 0000ffff 00009b00 SS =0000 00000000 0000ffff 00009300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=85 00 87 00 89 00 8b 00 00 00 86 00 88 00 8a 00 8c 00 00 90 <2e> 66 83 3e 50 73 00 0f 85 13 f3 31 c0 8e d0 66 bc 00 70 00 00 66 ba 08 3e 0f 00 e9 6e f1 dmesg of the host (real machine): [ 7125.538397] nested_vmx_exit_handled failed vm entry 7 More info: host: rhel7 kernel version: 3.10.0-123.el7.x86_64 guest: rhel7.1 kernel version: 3.10.0-221.el7.x86_64 Sorry. Looks like I have old kernel. Can be this bug cloned to rhel7 for information purposes? Hi. I was confused by: rule-engine: rhel-7.1.0 Ok. Thanks for info. Sorry for mess. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0290.html |
Description of problem: For wildcatpass host, qemu-kvm fail when booting L2 guest. QE tested the same scenarios on another Haswell(not wildcatpass) and SandyBridge host. L2 guest works well. so this issue maybe relevant to Hardware. Since wildcatpass host is newer host, and QE didn't hit this issue on another Haswell and SandyBridge host. so may not be a regression bug. Version-Release number of selected component (if applicable): 3.10.0-121.el7.x86_64 qemu-kvm-1.5.3-60.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. Enable nested virt on wildcatpass host # cat /sys/module/kvm_intel/parameters/nested Y # cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y # cat /sys/module/kvm_intel/parameters/ept Y # cat /sys/module/kvm_intel/parameters/enable_apicv Y 2.Boot RHEL7.0 L1 guest on host with '-cpu host' /usr/libexec/qemu-kvm -cpu host -M pc -enable-kvm -m 16G -smp 8,sockets=4,cores=2,threads=1 -name rhel7 -uuid a589bcc5-96e1-4d97-9e34-78954f2f5725 -nodefconfig -nodefaults -monitor stdio -rtc base=utc,clock=host,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive file=/root/guest/rhel7-0409.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=aa:54:00:11:22:33,bus=pci.0,addr=0x3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -qmp tcp:0:5566,server,nowait -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 3. Boot L2 RHEL7.0 guest on L1 with Haswell&SandyBridge&Westmere&Nehalem&Penryn Conroe&cpu64-rhel6 &default(without -cpu option) /usr/libexec/qemu-kvm -M pc -cpu Haswell -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -usb -device usb-tablet,id=input0 -name nested -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/rhel7-0409.qcow2,if=none,id=drive-virtio-disk,format=qcow2,aio=native,werror=stop,rerror=stop -device ide-drive,drive=drive-virtio-disk,id=virtio-disk,bootindex=1 -vnc :3 -monitor stdio -serial unix:/tmp/ttyS0,server,nowait 4. Actual results: (qemu) KVM: entry failed, hardware error 0x0 EAX=00000000 EBX=00000000 ECX=00000000 EDX=000006d3 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000e05b EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 000f0000 0000ffff 00009b00 SS =0000 00000000 0000ffff 00009300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=00 00 00 00 00 00 00 00 00 00 00 00 00 66 90 66 90 66 90 90 <2e> 66 83 3e 74 d1 00 0f 85 03 e5 31 c0 8e d0 66 bc 00 70 00 00 66 ba d5 41 0f 00 e9 7f e3 (qemu) info status VM status: paused (internal-error) (qemu) q Expected results: L2 guest works well Additional info: host info: 1.memory # free -g total used free shared buffers cached Mem: 31 5 25 0 0 3 -/+ buffers/cache: 2 29 Swap: 15 0 15 2. cpu(part info) #cat /proc/cpuinfo ........ processor : 55 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Genuine Intel(R) CPU @ 2.20GHz stepping : 1 microcode : 0x80000014 cpu MHz : 1634.359 cache size : 35840 KB physical id : 1 siblings : 28 core id : 14 cpu cores : 14 apicid : 61 initial apicid : 61 fpu : yes fpu_exception : yes cpuid level : 15 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm bogomips : 4395.09 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: 3. hostname intel-wildcatpass-02.khw.lab.eng.bos.redhat.com