RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1086058 - fail to boot L2 guest on wildcatpass Haswell host
Summary: fail to boot L2 guest on wildcatpass Haswell host
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel
Version: 7.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Bandan Das
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1116936
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-10 03:11 UTC by FuXiangChun
Modified: 2019-03-22 12:27 UTC (History)
11 users (show)

Fixed In Version: kernel-3.10.0-143.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-05 11:51:43 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0290 0 normal SHIPPED_LIVE Important: kernel security, bug fix, and enhancement update 2015-03-05 16:13:58 UTC

Description FuXiangChun 2014-04-10 03:11:46 UTC 
Description of problem:
For wildcatpass host, qemu-kvm fail when booting L2 guest. QE tested the same scenarios on another Haswell(not wildcatpass) and SandyBridge host. L2 guest works well. so this issue maybe relevant to Hardware. Since wildcatpass host is newer host, and QE didn't hit this issue on another Haswell and SandyBridge host. so may not be a regression bug.

Version-Release number of selected component (if applicable):
3.10.0-121.el7.x86_64
qemu-kvm-1.5.3-60.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Enable nested virt on wildcatpass host

# cat /sys/module/kvm_intel/parameters/nested
Y
# cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs
Y
# cat /sys/module/kvm_intel/parameters/ept
Y
# cat /sys/module/kvm_intel/parameters/enable_apicv
Y

2.Boot RHEL7.0 L1 guest on host with '-cpu host'

/usr/libexec/qemu-kvm -cpu host -M pc -enable-kvm -m 16G -smp 8,sockets=4,cores=2,threads=1 -name rhel7 -uuid a589bcc5-96e1-4d97-9e34-78954f2f5725  -nodefconfig -nodefaults -monitor stdio -rtc base=utc,clock=host,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive file=/root/guest/rhel7-0409.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0  -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=aa:54:00:11:22:33,bus=pci.0,addr=0x3  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -qmp tcp:0:5566,server,nowait -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432

3. Boot L2 RHEL7.0 guest on L1 with Haswell&SandyBridge&Westmere&Nehalem&Penryn Conroe&cpu64-rhel6 &default(without -cpu option)

/usr/libexec/qemu-kvm -M pc -cpu Haswell -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -usb -device usb-tablet,id=input0 -name nested -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/rhel7-0409.qcow2,if=none,id=drive-virtio-disk,format=qcow2,aio=native,werror=stop,rerror=stop -device ide-drive,drive=drive-virtio-disk,id=virtio-disk,bootindex=1 -vnc :3  -monitor stdio -serial unix:/tmp/ttyS0,server,nowait

4.

Actual results:
(qemu) KVM: entry failed, hardware error 0x0
EAX=00000000 EBX=00000000 ECX=00000000 EDX=000006d3
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000e05b EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 66 90 66 90 66 90 90 <2e> 66 83 3e 74 d1 00 0f 85 03 e5 31 c0 8e d0 66 bc 00 70 00 00 66 ba d5 41 0f 00 e9 7f e3

(qemu) info status
VM status: paused (internal-error)
(qemu) q

Expected results:
L2 guest works well

Additional info:
host info:
1.memory
# free -g
             total       used       free     shared    buffers     cached
Mem:            31          5         25          0          0          3
-/+ buffers/cache:          2         29
Swap:           15          0         15

2. cpu(part info)
#cat /proc/cpuinfo

........
processor	: 55
vendor_id	: GenuineIntel
cpu family	: 6
model		: 63
model name	: Genuine Intel(R) CPU @ 2.20GHz
stepping	: 1
microcode	: 0x80000014
cpu MHz		: 1634.359
cache size	: 35840 KB
physical id	: 1
siblings	: 28
core id		: 14
cpu cores	: 14
apicid		: 61
initial apicid	: 61
fpu		: yes
fpu_exception	: yes
cpuid level	: 15
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm
bogomips	: 4395.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

3. hostname
intel-wildcatpass-02.khw.lab.eng.bos.redhat.com
Comment 2 FuXiangChun 2014-08-28 07:08:33 UTC 
Re-tested this bug with 3.10.0-145.el7.x86_64 (host & guest) and qemu-kvm-rhev-2.1.0-2.el7.x86_64 on host intel-wildcatpass-02.khw.lab.eng.bos.redhat.com.

1. Enable nested virt on wildcatpass host

# cat /sys/module/kvm_intel/parameters/nested
Y
# cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs
Y
# cat /sys/module/kvm_intel/parameters/ept
Y
# cat /sys/module/kvm_intel/parameters/enable_apicv
Y

2.Boot RHEL7.1 L1 guest on host with '-cpu host'

3.Boot L2 RHEL7.0 guest on L1 with Haswell

result:

1. L0 host show call trace. 

[ 2767.750777] WARNING: at arch/x86/kvm/vmx.c:8686 nested_vmx_vmexit+0x282/0x290 [kvm_intel]()
[ 2767.750789] Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm sg bridge stp llc intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ixgbe ptp aesni_intel pps_core lrw iTCO_wdt mdio gf128mul glue_helper iTCO_vendor_support dca ablk_helper mei_me lpc_ich cryptd pcspkr i2c_i801 mfd_core mei shpchp wmi ipmi_si ipmi_msghandler acpi_pad acpi_power_meter mperf xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ahci ttm libahci drm libata i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kvm]
[ 2767.750790] CPU: 26 PID: 3659 Comm: qemu-kvm Tainted: G        W   --------------   3.10.0-145.el7.x86_64 #1
[ 2767.750791] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0037.D10.1408060946 08/06/2014
[ 2767.750795]  0000000000000000 00000000085cacc2 ffff88086728fc60 ffffffff815eb72b
[ 2767.750797]  ffff88086728fc98 ffffffff8106a6c1 ffff880386090000 ffff8802cf661000
[ 2767.750799]  0000000000000001 ffff880864eb4440 ffff880386090030 ffff88086728fca8
[ 2767.750799] Call Trace:
[ 2767.750802]  [<ffffffff815eb72b>] dump_stack+0x19/0x1b
[ 2767.750804]  [<ffffffff8106a6c1>] warn_slowpath_common+0x61/0x80
[ 2767.750806]  [<ffffffff8106a7ea>] warn_slowpath_null+0x1a/0x20
[ 2767.750808]  [<ffffffffa046ddb2>] nested_vmx_vmexit+0x282/0x290 [kvm_intel]
[ 2767.750810]  [<ffffffffa0465e48>] ? vmx_sync_pir_to_irr+0x28/0x30 [kvm_intel]
[ 2767.750812]  [<ffffffffa046dec0>] vmx_check_nested_events+0x100/0x120 [kvm_intel]
[ 2767.750819]  [<ffffffffa04e8d2f>] vcpu_enter_guest+0x42f/0xd50 [kvm]
[ 2767.750821]  [<ffffffffa0469d6b>] ? vmx_deliver_posted_interrupt+0x3b/0x80 [kvm_intel]
[ 2767.750827]  [<ffffffffa0508722>] ? __apic_accept_irq+0x162/0x2b0 [kvm]
[ 2767.750833]  [<ffffffffa04ee848>] kvm_arch_vcpu_ioctl_run+0x208/0x480 [kvm]
[ 2767.750837]  [<ffffffffa04d8e3a>] kvm_vcpu_ioctl+0x2aa/0x580 [kvm]
[ 2767.750839]  [<ffffffff812015d1>] ? fsnotify+0x251/0x350
[ 2767.750841]  [<ffffffff811d2765>] do_vfs_ioctl+0x2e5/0x4c0
[ 2767.750843]  [<ffffffff812672de>] ? file_has_perm+0xae/0xc0
[ 2767.750845]  [<ffffffff811d29e1>] SyS_ioctl+0xa1/0xc0
[ 2767.750847]  [<ffffffff815fbc59>] system_call_fastpath+0x16/0x1b
[ 2767.750847] ---[ end trace a300483cabb59779 ]---
[ 2767.915233] WARNING: at arch/x86/kvm/vmx.c:8686 nested_vmx_vmexit+0x282/0x290 [kvm_intel]()
[ 2767.915265] Modules linked in: vhost_net macvtap macvlan tun kvm_intel kvm sg bridge stp llc intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ixgbe ptp aesni_intel pps_core lrw iTCO_wdt mdio gf128mul glue_helper iTCO_vendor_support dca ablk_helper mei_me lpc_ich cryptd pcspkr i2c_i801 mfd_core mei shpchp wmi ipmi_si ipmi_msghandler acpi_pad acpi_power_meter mperf xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ahci ttm libahci drm libata i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kvm]
[ 2767.915268] CPU: 26 PID: 3659 Comm: qemu-kvm Tainted: G        W   --------------   3.10.0-145.el7.x86_64 #1
[ 2767.915269] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0037.D10.1408060946 08/06/2014
[ 2767.915277]  0000000000000000 00000000085cacc2 ffff88086728fc60 ffffffff815eb72b
[ 2767.915282]  ffff88086728fc98 ffffffff8106a6c1 ffff880386090000 ffff8802cf661000
[ 2767.915288]  0000000000000001 ffff880864eb4440 ffff880386090030 ffff88086728fca8
[ 2767.915289] Call Trace:
[ 2767.915296]  [<ffffffff815eb72b>] dump_stack+0x19/0x1b
[ 2767.915300]  [<ffffffff8106a6c1>] warn_slowpath_common+0x61/0x80
[ 2767.915304]  [<ffffffff8106a7ea>] warn_slowpath_null+0x1a/0x20
[ 2767.915311]  [<ffffffffa046ddb2>] nested_vmx_vmexit+0x282/0x290 [kvm_intel]
[ 2767.915316]  [<ffffffffa0465e48>] ? vmx_sync_pir_to_irr+0x28/0x30 [kvm_intel]
[ 2767.915322]  [<ffffffffa046dec0>] vmx_check_nested_events+0x100/0x120 [kvm_intel]
[ 2767.915339]  [<ffffffffa04e8d2f>] vcpu_enter_guest+0x42f/0xd50 [kvm]
[ 2767.915345]  [<ffffffffa0469d6b>] ? vmx_deliver_posted_interrupt+0x3b/0x80 [kvm_intel]
[ 2767.915363]  [<ffffffffa0508722>] ? __apic_accept_irq+0x162/0x2b0 [kvm]
[ 2767.915379]  [<ffffffffa04ee848>] kvm_arch_vcpu_ioctl_run+0x208/0x480 [kvm]
[ 2767.915391]  [<ffffffffa04d8e3a>] kvm_vcpu_ioctl+0x2aa/0x580 [kvm]
[ 2767.915395]  [<ffffffff812015d1>] ? fsnotify+0x251/0x350
[ 2767.915413]  [<ffffffff811d2765>] do_vfs_ioctl+0x2e5/0x4c0
[ 2767.915415]  [<ffffffff812672de>] ? file_has_perm+0xae/0xc0
[ 2767.915417]  [<ffffffff811d29e1>] SyS_ioctl+0xa1/0xc0
[ 2767.915419]  [<ffffffff815fbc59>] system_call_fastpath+0x16/0x1b
[ 2767.915419] ---[ end trace a300483cabb5977a ]---

2. L2 guest boot fail(black screen)
Comment 3 FuXiangChun 2014-08-28 07:17:44 UTC 
As original problem has been solved.  For host call trace issue, Form QE POV, It is another issue. In order to clearly trace it. QE filed another bug 1134700.  

BTW, If you think should DUP this two bugs. then re-assign this bug. For QE, It is not problem.
Comment 6 Miroslav Rezanina 2014-09-01 06:20:54 UTC 
=Reasign to kernel as this is not qemu-kvm issue
Comment 8 FuXiangChun 2014-11-26 05:33:38 UTC 
Re-tested this issue with 3.10.0-208.el7.x86_64 and qemu-kvm-rhev-2.1.2-13.el7.x86_64 on wildcatpass Haswell host.

when booting L2 guest on L1 guest.

result:
error message is gong as comment 0, But Host call trace(bug 1134700). according to this test result, QE think this bug can be verified. we will continue to concern bug 1134700.
Comment 9 juzhang 2014-12-02 04:03:10 UTC 
According to comment8, set this issue as verified.
Comment 10 Robin Hack 2015-01-09 12:10:36 UTC 
Hi. All. I have still troubles on:
Host CPU (real machine):
processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
stepping	: 4
microcode	: 0x428
cpu MHz		: 2362.582
cache size	: 15360 KB
physical id	: 1
siblings	: 12
core id		: 5
cpu cores	: 6
apicid		: 43
initial apicid	: 43
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips	: 4204.45
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

Guest - this guest is host for virtual machine which I want to run:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
stepping	: 4
microcode	: 0x1
cpu MHz		: 2099.998
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm xsaveopt fsgsbase tsc_adjust smep erms
bogomips	: 4199.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

/usr/libexec/qemu-kvm -M pc -cpu Haswell -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -usb -device usb-tablet,id=input0 -name nested -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew  -vnc :3  -monitor stdio -serial unix:/tmp/ttyS0,server,nowait
QEMU 1.5.3 monitor - type 'help' for more information
(qemu) KVM: entry failed, hardware error 0x0
EAX=00000000 EBX=00000000 ECX=00000000 EDX=000306c1
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000e05b EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=85 00 87 00 89 00 8b 00 00 00 86 00 88 00 8a 00 8c 00 00 90 <2e> 66 83 3e 50 73 00 0f 85 13 f3 31 c0 8e d0 66 bc 00 70 00 00 66 ba 08 3e 0f 00 e9 6e f1


dmesg of the host (real machine):
[ 7125.538397] nested_vmx_exit_handled failed vm entry 7
Comment 11 Robin Hack 2015-01-09 12:23:00 UTC 
More info:
host: rhel7 kernel version: 3.10.0-123.el7.x86_64
guest: rhel7.1 kernel version: 3.10.0-221.el7.x86_64
Comment 12 Robin Hack 2015-01-09 13:51:59 UTC 
Sorry. Looks like I have old kernel. Can be this bug cloned to rhel7 for information purposes?
Comment 14 Robin Hack 2015-01-09 14:58:01 UTC 
Hi. I was confused by: rule-engine: 	rhel-7.1.0

Ok. Thanks for info.

Sorry for mess.
Comment 16 errata-xmlrpc 2015-03-05 11:51:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0290.html
Note You need to log in before you can comment on or make changes to this bug.