Bug 1063124
| Summary: | Fail to boot windows 2012/win8.1/win8 x86_64 guest with '-cpu SandyBridge/Opteron_G4' with hv_relaxed flag | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Xu Han <xuhan> | ||||||||
| Component: | qemu-kvm | Assignee: | Vadim Rozenfeld <vrozenfe> | ||||||||
| Status: | CLOSED WONTFIX | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||||
| Severity: | high | Docs Contact: | Jiri Herrmann <jherrman> | ||||||||
| Priority: | high | ||||||||||
| Version: | 6.6 | CC: | ailan, chayang, cjg9411, coli, ehabkost, jbelka, juzhang, knoel, ldelouw, michen, mkenneth, ngu, pbonzini, qzhang, rbalakri, ricky.schneberger, rkrcmar, rpacheco, virt-bugs, virt-maint, vrozenfe, xfu, xuhan, xuwei, yfu, yiwei, zhguo | ||||||||
| Target Milestone: | rc | Keywords: | Reopened | ||||||||
| Target Release: | --- | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Known Issue | |||||||||
| Doc Text: |
Guests with recent Windows systems in some cases fail to boot if "hv_relaxed" is used
Attempting to boot KVM guests with the following operating systems currently fails with an "error code: 0x0000001E" message if the value of the "-cpu" option is `SandyBridge` or `Opteron_G4` and the "hv_relaxed" option is used.
* 64-bit Windows 8 or later
* 64-bit Windows Server 2012 or later
To work around this problem, do not use "hv_relaxed".
|
Story Points: | --- | ||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2017-01-26 15:23:00 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Bug Depends On: | |||||||||||
| Bug Blocks: | 1134550, 1269194, 1359965 | ||||||||||
| Attachments: |
|
||||||||||
|
Description
Xu Han
2014-02-10 05:03:52 UTC
Created attachment 861232 [details]
screen shot of errors
Does it work well with '-cpu SandyBridge' only, without hv_relaxed flag? Thanks, Vadim. (In reply to Vadim Rozenfeld from comment #3) > Does it work well with '-cpu SandyBridge' only, without hv_relaxed flag? It works well with '-cpu SandyBridge' only. e.g: # qemu-kvm \ -nodefaults -M rhel6.5.0 -m 2G \ -cpu SandyBridge \ -smp 4,threads=2,cores=2,sockets=1 -nodefconfig -vga cirrus -vnc :0 \ -cdrom /root/en_windows_server_2012_x64_dvd_915478.iso But happened with 'hv_relaxed' flag append. qemu-kvm-0.12.1.2-2.424.el6.x86_64 kernel-2.6.32-459.el6.x86_64 (In reply to Xu Han from comment #4) > (In reply to Vadim Rozenfeld from comment #3) > > Does it work well with '-cpu SandyBridge' only, without hv_relaxed flag? > > It works well with '-cpu SandyBridge' only. e.g: > # qemu-kvm \ > -nodefaults -M rhel6.5.0 -m 2G \ > -cpu SandyBridge \ > -smp 4,threads=2,cores=2,sockets=1 -nodefconfig -vga cirrus -vnc :0 \ > -cdrom /root/en_windows_server_2012_x64_dvd_915478.iso > > But happened with 'hv_relaxed' flag append. > > qemu-kvm-0.12.1.2-2.424.el6.x86_64 > kernel-2.6.32-459.el6.x86_64 thanks for your prompt reply. Vadim Also hit this issue on Win8.0 and Win8.1 64 bit guest. The 32 bit version works well with "-cpu 'SandyBridge',+sep,hv_relaxed". Since it doesn't happen in RHEL7, maybe some kind of bisect can help. Vadim, do you suspect any specific RHEL7 patches that fixed it? (In reply to Ronen Hod from comment #7) > Since it doesn't happen in RHEL7, maybe some kind of bisect can help. Vadim, > do you suspect any specific RHEL7 patches that fixed it? Probably yes. Let's go through cpu flags to check if there is any difference between RHEL6.6 and RHEL7 *** Bug 1069082 has been marked as a duplicate of this bug. *** Also hit this issue with "-cpu 'Opteron_G4',hv_relaxed", remove the hv_relaxed flag, works well. Can we try reproducing the problem on rhel 7.2? (In reply to Vadim Rozenfeld from comment #13) > Can we try reproducing the problem on rhel 7.2? Not hit this issue on rhel 7.2: SandyBridge platform: (windows2012/win8.1) kernel-3.10.0-327.el7.x86_64 qemu-kvm-rhev-2.3.0-31.el7 qemu-kvm-1.5.3-105.el7 Opteron_G4 platform: (windows2012/win8.1) kernel-3.10.0-327.el7.x86_64 qemu-kvm-rhev-2.3.0-31.el7 qemu-kvm-1.5.3-105.el7 (In reply to Xueqiang Wei from comment #14) > (In reply to Vadim Rozenfeld from comment #13) > > Can we try reproducing the problem on rhel 7.2? > > > Not hit this issue on rhel 7.2: > SandyBridge platform: (windows2012/win8.1) > kernel-3.10.0-327.el7.x86_64 > qemu-kvm-rhev-2.3.0-31.el7 > qemu-kvm-1.5.3-105.el7 > > Opteron_G4 platform: (windows2012/win8.1) > kernel-3.10.0-327.el7.x86_64 > qemu-kvm-rhev-2.3.0-31.el7 > qemu-kvm-1.5.3-105.el7 Thank you. Closing the bug, based on this comment. (In reply to Vadim Rozenfeld from comment #15) > (In reply to Xueqiang Wei from comment #14) > > (In reply to Vadim Rozenfeld from comment #13) > > > Can we try reproducing the problem on rhel 7.2? > > > > > > Not hit this issue on rhel 7.2: > > SandyBridge platform: (windows2012/win8.1) > > kernel-3.10.0-327.el7.x86_64 > > qemu-kvm-rhev-2.3.0-31.el7 > > qemu-kvm-1.5.3-105.el7 > > > > Opteron_G4 platform: (windows2012/win8.1) > > kernel-3.10.0-327.el7.x86_64 > > qemu-kvm-rhev-2.3.0-31.el7 > > qemu-kvm-1.5.3-105.el7 > > Thank you. > Closing the bug, based on this comment. Vadim, Would you please tell more why this bug is closed with resolution CURRENTRELEASE? This is bug against RHEL 6 product and apparently this bug is reproducible on RHEL 6 per latest Comment 11. (In reply to Chao Yang from comment #16) > (In reply to Vadim Rozenfeld from comment #15) > > (In reply to Xueqiang Wei from comment #14) > > > (In reply to Vadim Rozenfeld from comment #13) > > > > Can we try reproducing the problem on rhel 7.2? > > > > > > > > > Not hit this issue on rhel 7.2: > > > SandyBridge platform: (windows2012/win8.1) > > > kernel-3.10.0-327.el7.x86_64 > > > qemu-kvm-rhev-2.3.0-31.el7 > > > qemu-kvm-1.5.3-105.el7 > > > > > > Opteron_G4 platform: (windows2012/win8.1) > > > kernel-3.10.0-327.el7.x86_64 > > > qemu-kvm-rhev-2.3.0-31.el7 > > > qemu-kvm-1.5.3-105.el7 > > > > Thank you. > > Closing the bug, based on this comment. > > Vadim, > > Would you please tell more why this bug is closed with resolution > CURRENTRELEASE? This is bug against RHEL 6 product and apparently this bug > is reproducible on RHEL 6 per latest Comment 11. My fault, bringing it back to assigned. Btw, how does it work on rhel 6.8? Thanks, Vadim. Hi Zhiyi, Could you help a try and reply to Vadim's questions above? Issue can be reproduced on rhel 6.8 and environment briefs: Platforms: Opteron_G4/SandyBridge Os:win2012 x64/win8.1 x64 kernel&qemu:2.6.32-583.el6.x86_64, qemu-kvm-0.12.1.2-2.481.el6.x86_64 Issue cannot be reproduced in same environment but change Os to win8.1 x86 version. Error prompt: error code: 0x0000001E (In reply to Guo, Zhiyi from comment #19) > Issue can be reproduced on rhel 6.8 and environment briefs: > Platforms: Opteron_G4/SandyBridge > Os:win2012 x64/win8.1 x64 > kernel&qemu:2.6.32-583.el6.x86_64, qemu-kvm-0.12.1.2-2.481.el6.x86_64 > > Issue cannot be reproduced in same environment but change Os to win8.1 x86 > version. > > Error prompt: error code: 0x0000001E Thanks, Can you please share a screenshot, taken when the BSOD happens? Vadim. Created attachment 1092621 [details]
screenshot of BSOD on rhel 6.8
Xu Han has attached the screenshot for BSOD, attach screenshot for rhel 6.8
(In reply to Guo, Zhiyi from comment #21) > Created attachment 1092621 [details] > screenshot of BSOD on rhel 6.8 > > Xu Han has attached the screenshot for BSOD, attach screenshot for rhel 6.8 Thanks. Now, can you please run the same VM without hv_relaxed flag, trigger BSOD with NMI, and then share generated crash dump file? Vadim. Upload crash dump file to http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/rhel6/bug1063124/ Include Opteron_G4.dmp and SandyBridge.DMP The bug could be reproduced on win8.1 x86_64 and win10 x86_64 guests with "-cpu 'Haswell',hv_relaxed", please refer to following bug comment for details: https://bugzilla.redhat.com/show_bug.cgi?id=1204453#c25 The sw versions are as follows: Host kernel: 2.6.32-595.el6.x86_64 Qemu-kvm-rhev: qemu-kvm-rhev-0.12.1.2-2.483.el6.x86_64 Virtio-win: virtio-win-1.7.5-0.el6.iso (In reply to Gu Nini from comment #24) > The bug could be reproduced on win8.1 x86_64 and win10 x86_64 guests with > "-cpu 'Haswell',hv_relaxed", please refer to following bug comment for > details: > > https://bugzilla.redhat.com/show_bug.cgi?id=1204453#c25 > > The sw versions are as follows: > Host kernel: 2.6.32-595.el6.x86_64 > Qemu-kvm-rhev: qemu-kvm-rhev-0.12.1.2-2.483.el6.x86_64 > Virtio-win: virtio-win-1.7.5-0.el6.iso Failed to reproduce the bug on Win7.sp1_x86_64 guest on the same host. can we try it with ignore_msrs kvm's parameter turned on? Thanks you. Vadim. Hi Vadim,
Reproduced this issue.
Version:
qemu-kvm:qemu-kvm-0.12.1.2-2.415.el6.x86_64
kernel:kernel-2.6.32-663.el6.x86_64
guest:win2012 64bit
--------------------------------------
Also reproduced this issue with ignore_msrs=Y
The way of turning on ignore_msrs:
echo 1 > /sys/module/kvm/parameters/ignore_msrs
echo "options kvm ignore_msrs=1" | sudo tee /etc/modprobe.d/kvm-ignore-msrs.conf
Test results:
boot failed.
Additional info:
1.boot win2012 guest with '-cpu SandyBridge,hv_relaxed'
/usr/libexec/qemu-kvm \
-nodefaults \
-M rhel6.5.0 \
-m 4G \
-cpu SandyBridge,hv_relaxed \
-smp 8,threads=2,cores=2,sockets=2 \
-nodefconfig \
-qmp tcp:0:5550,server,nowait \
-rtc base=utc,clock=host,driftfix=slew \
-vga cirrus \
-boot order=cd,menu=on \
-vnc :0 \
-monitor stdio \
-drive file=/home/win2012/64bit/bug-1063124,if=none,id=guest-img,cache=none,aio=native,rerror=stop,werror=stop \
-device ide-drive,drive=guest-img,id=os-disk,bootindex=1 \
-cdrom /home/win2012/64bit/en_windows_server_2012_x64_dvd_915478.iso \
2.# cat /proc/cpuinfo
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 42
model name : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping : 7
microcode : 41
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips : 6784.63
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
3.# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 42
Model name: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Stepping: 7
CPU MHz: 1600.000
BogoMIPS: 6784.63
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Created attachment 1220284 [details]
screen shot of errors
(In reply to Yiqian Wei from comment #33) > Hi Vadim, > Reproduced this issue. > Version: > qemu-kvm:qemu-kvm-0.12.1.2-2.415.el6.x86_64 > kernel:kernel-2.6.32-663.el6.x86_64 > guest:win2012 64bit > -------------------------------------- > Also reproduced this issue with ignore_msrs=Y > The way of turning on ignore_msrs: > echo 1 > /sys/module/kvm/parameters/ignore_msrs > echo "options kvm ignore_msrs=1" | sudo tee > /etc/modprobe.d/kvm-ignore-msrs.conf > > Test results: > boot failed. > > Additional info: > 1.boot win2012 guest with '-cpu SandyBridge,hv_relaxed' > /usr/libexec/qemu-kvm \ > -nodefaults \ > -M rhel6.5.0 \ > -m 4G \ > -cpu SandyBridge,hv_relaxed \ > -smp 8,threads=2,cores=2,sockets=2 \ > -nodefconfig \ > -qmp tcp:0:5550,server,nowait \ > -rtc base=utc,clock=host,driftfix=slew \ > -vga cirrus \ > -boot order=cd,menu=on \ > -vnc :0 \ > -monitor stdio \ > -drive > file=/home/win2012/64bit/bug-1063124,if=none,id=guest-img,cache=none, > aio=native,rerror=stop,werror=stop \ > -device ide-drive,drive=guest-img,id=os-disk,bootindex=1 \ > -cdrom /home/win2012/64bit/en_windows_server_2012_x64_dvd_915478.iso \ > 2.# cat /proc/cpuinfo > processor : 7 > vendor_id : GenuineIntel > cpu family : 6 > model : 42 > model name : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz > stepping : 7 > microcode : 41 > cpu MHz : 1600.000 > cache size : 8192 KB > physical id : 0 > siblings : 8 > core id : 3 > cpu cores : 4 > apicid : 7 > initial apicid : 7 > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat > pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm > constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf > pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm > pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm > ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid > bogomips : 6784.63 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: > 3.# lscpu > Architecture: x86_64 > CPU op-mode(s): 32-bit, 64-bit > Byte Order: Little Endian > CPU(s): 8 > On-line CPU(s) list: 0-7 > Thread(s) per core: 2 > Core(s) per socket: 4 > Socket(s): 1 > NUMA node(s): 1 > Vendor ID: GenuineIntel > CPU family: 6 > Model: 42 > Model name: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz > Stepping: 7 > CPU MHz: 1600.000 > BogoMIPS: 6784.63 > Virtualization: VT-x > L1d cache: 32K > L1i cache: 32K > L2 cache: 256K > L3 cache: 8192K > NUMA node0 CPU(s): 0-7 Retest----the latest version of qemu-kvm host version: qemu-kvm-0.12.1.2-2.496.el6.x86_64 kernel-2.6.32-663.el6.x86_64 -------------------- Test results: boot failed. Comment from Vadim: I managed to reproduce and partially debug this issue.The crash itself happens inside of HvlEnlightenProcessorfunction when kernel tries to access unsupported Hyper-VMSRs. The simplest work around for this issue will notto specify hv_relaxed flag at all for all Windows gueststarting form Win8/WS2012. Recent Windows platforms will deactivate 101 BSOD if hypervisor flag is present (even if hv_relaxed has not been specified).Vadim. Based on comment#36 let's update our docs to not use the hv_relaxed in RHEL6 for Windows8/2012 and newer. QE, can you verifiy it by running stress tests and check that you do not get BSOD 101 with Windows8/2012/10/2016 when the hv_relaxed is not set. (In reply to Amnon Ilan from comment #37) > Based on comment#36 let's update our docs to not use the hv_relaxed in RHEL6 > for Windows8/2012 and newer. > > QE, can you verifiy it by running stress tests and check that you do not get > BSOD 101 with Windows8/2012/10/2016 when the hv_relaxed is not set. Hi Yiwei, Could you have a try? Best Regards, Junyi test version qemu-kvm:qemu-kvm-0.12.1.2-2.501.el6.x86_64 kernel:kernel-2.6.32-663.el6.x86_64 virtio-win:virtio-win-1.8.0-1.el6.noarch guest:win2012 64bit a.boot win2012 guest with '-cpu SandyBridge' and run stress tests in guest. Test results: guest not get BSOD 101. b.boot win2012 guest with '-cpu SandBridge,hv_relaxed' Test results: boot failed. Additional info: 1./usr/libexec/qemu-kvm \ -nodefaults \ -M rhel6.5.0 \ -m 4G \ -cpu SandyBridge,enforce \ -smp 8,threads=2,cores=2,sockets=2 \ -nodefconfig \ -qmp tcp:0:5551,server,nowait \ -rtc base=utc,clock=host,driftfix=slew \ -vga cirrus \ -boot order=cd,menu=on \ -vnc :1 \ -usbdevice tablet \ -monitor stdio \ -drive file=/home/win2012/64bit/bz1063124,if=none,id=drive-virtio-disk0,format=qcow2,cache=none \ -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ -netdev tap,id=hostnet0,vhost=on \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:aa:c3:8a,bus=pci.0,addr=0x8 \ -fda /usr/share/virtio-win/virtio-win_amd64.vfd \ |