Description of problem: Boot Win2012r2/win7-64bit guest with "-cpu Haswell/SandyBridge,hv_vapic" on IvyBridge host, the guests cannot boot successfully. Version-Release number of selected component (if applicable): kernel-3.10.0-121.el7.x86_64 qemu-kvm-rhev-1.5.3-60.el7ev.x86_64 How reproducible: 100% Steps to Reproduce: 1. Boot a win2012r2/win7-64 guest with "+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time" # /usr/libexec/qemu-kvm -cpu Haswell,+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -name rhel7 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/test1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.com,bus=virtio-serial0.0,id=port1 -drive file=/home/win7-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-netpci0,mac=54:52:1b:35:2c:01,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -serial unix:/tmp/ttyS0,server,nowait -spice port=5900,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -monitor stdio -qmp tcp:0:4445,server,nowait -vnc :1 2. 3. Actual results: after step1, guest cannot boot and the scrrenshot, please refer to the attchment file. Expected results: guest can boot successfully. Additional info: 1. on SandyBridge host, use the same command line , win2012r2 and win7-64 guest can boot successfully. 2. on amd Opteron_G2 host, use the same command line , win2012r2 and win7-64 guest can boot successfully. 2. on IvyBridge host, only with "-cpu Haswell/SandyBridge,+x2apic,hv_vapic", the guests cannot boot.
Created attachment 890345 [details] guest screenshot when booting failed
The cpu info of IvyBridge as following: # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 120 On-line CPU(s) list: 0-119 Thread(s) per core: 2 Core(s) per socket: 15 Socket(s): 4 NUMA node(s): 4 Vendor ID: GenuineIntel CPU family: 6 Model: 62 Model name: Intel(R) Xeon(R) CPU E7-4890 v2 @ 2.80GHz Stepping: 7 CPU MHz: 3232.468 BogoMIPS: 5595.32 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 38400K NUMA node0 CPU(s): 0-14,60-74 NUMA node1 CPU(s): 15-29,75-89 NUMA node2 CPU(s): 30-44,90-104 NUMA node3 CPU(s): 45-59,105-119 The info of /proc/cpuinfo of IvyBridge as following: # cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 62 model name : Intel(R) Xeon(R) CPU E7-4890 v2 @ 2.80GHz stepping : 7 microcode : 0x700 cpu MHz : 3199.984 cache size : 38400 KB physical id : 0 siblings : 30 core id : 0 cpu cores : 15 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms bogomips : 5586.19 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual
> 2. on IvyBridge host, only with "-cpu Haswell/SandyBridge,+x2apic,hv_vapic", > the guests cannot boot. Can we try without specifying +x2apic flag for Haswell/SandyBridge, they both should have it be default. Thanks, Vadim.
(In reply to Vadim Rozenfeld from comment #4) > > 2. on IvyBridge host, only with "-cpu Haswell/SandyBridge,+x2apic,hv_vapic", > > the guests cannot boot. > > Can we try without specifying +x2apic flag for Haswell/SandyBridge, they both > should have it be default. > > Thanks, > Vadim. Hi, Vadim, On ivybridge host, I use "-cpu Haswell,hv_vapic" to boot a win2012r2 guest. It is also failed and the sceenshot is also as comment 1. The ivybridge host supports apicv: # cat /sys/module/kvm_intel/parameters/enable_apicv Y Disable apicv on this host as following commands, use "-cpu Haswell,hv_vapic" to boot a win2012r2 guest, the guest can boot successfully. # modprobe -r kvm_intel # modprobe kvm_intel enable_apicv=N # cat /sys/module/kvm_intel/parameters/enable_apicv N Thanks. Huiqing
I think I understand what's going on here. When Hyper-V enlightenments are in effect, Windows prefers to issue an Hyper-V MSR write to issue an EOI rather than an x2apic MSR write. The Hyper-V MSR write is not handled by the processor, and on EOI the processor will modify the highest in-service interrupt (SVI) field of the VMCS, as explained in section 29.1.4 of the SDM. We need to do the same on Hyper-V MSR writes. Can I have a test environment where I can test a fix?
> Can I have a test environment where I can test a fix? Hi Huding, Can we provide the host which can be stably reproduced this issue? Best Regards, Junyi
huiqingding, where can I find the Windows 2012 ISO? Could you copy it for me to /root? Thanks!
Created attachment 897538 [details] tested patch Brew build id 7474234
Reproduced this bug with <kernel 3.10.0-143.el7.x86_64. result: win2k8r2 guest black screen. verified this bug with 3.10.0-143.el7.x86_64. guest works well.
*** Bug 1129225 has been marked as a duplicate of this bug. ***
Test this bug on an Ivybridge host using the following version: kernel-3.10.0-146.el7.x86_64 qemu-kvm-rhev-2.1.0-2.el7.x86_64 The host supports apicv: # cat /sys/module/kvm_intel/parameters/enable_apicv Y Steps to test: 1. boot a win2k8r2 guest with "+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time" # /usr/libexec/qemu-kvm -M pc -cpu Haswell,+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -name rhel7 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0 -chardev socket,id=channel1,path=/tmp/test1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.com,bus=virtio-serial0.0,id=port1 -drive file=/home/win2k8r2.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -serial unix:/tmp/ttyS0,server,nowait -spice port=5900,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -monitor stdio -qmp tcp:0:4445,server,nowait -nodefaults -boot menu=on -cdrom 7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso -drive file=/home/driver.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,unit=1,ver=fuxc-ver-cdrom,bus=ide.0,unit=1 Actual results: after step1, the guest can boot normally.
Moving to correct component
Reproduce this bug using the following version: kernel-3.10.0-142.el7.x86_64 qemu-kvm-1.5.3-77.el7.x86_64 The host supports apicv: # cat /sys/module/kvm_intel/parameters/enable_apicv Y Steps to Reproduce: 1. boot a win2012r2 guest with "+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time" # /usr/libexec/qemu-kvm -M pc -cpu Haswell,+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -name rhel7 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0 -chardev socket,id=channel1,path=/tmp/test1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.com,bus=virtio-serial0.0,id=port1 -drive file=/home/win2012r2-blk.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -serial unix:/tmp/ttyS0,server,nowait -spice port=5900,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -monitor stdio -qmp tcp:0:4445,server,nowait -nodefaults -boot menu=on -cdrom 9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_EN-US-IR3_SSS_X64FREE_EN-US_DV9.ISO -drive file=/home/driver.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,unit=1,ver=fuxc-ver-cdrom,bus=ide.0,unit=1 Actual results: after step1, the guest cannot boot sucessfully.
Test this issue using the folliwng version: kernel-3.10.0-196.el7.x86_64 qemu-kvm-1.5.3-77.el7.x86_64 Use the same command line of comment 26 to boot a win2012r2 guest and the result is ok: the guest can boot normally.
Test this issue using the folliwng version: kernel-3.10.0-196.el7.x86_64 qemu-kvm-rhev-2.1.2-6.el7.x86_64 Use the same command line of comment 26 to boot a win2012r2 guest and the result is ok: the guest can boot normally.
According to comment26-comment28, set this issue as verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0290.html