Bug 1091818

Created attachment 890345 [details]
guest screenshot when booting failed

The cpu info of IvyBridge as following:
# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                120
On-line CPU(s) list:   0-119
Thread(s) per core:    2
Core(s) per socket:    15
Socket(s):             4
NUMA node(s):          4
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Model name:            Intel(R) Xeon(R) CPU E7-4890 v2 @ 2.80GHz
Stepping:              7
CPU MHz:               3232.468
BogoMIPS:              5595.32
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              38400K
NUMA node0 CPU(s):     0-14,60-74
NUMA node1 CPU(s):     15-29,75-89
NUMA node2 CPU(s):     30-44,90-104
NUMA node3 CPU(s):     45-59,105-119

The info of /proc/cpuinfo of IvyBridge as following:
# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E7-4890 v2 @ 2.80GHz
stepping        : 7
microcode       : 0x700
cpu MHz         : 3199.984
cache size      : 38400 KB
physical id     : 0
siblings        : 30
core id         : 0
cpu cores       : 15
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 5586.19
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual

> 2. on IvyBridge host, only with "-cpu Haswell/SandyBridge,+x2apic,hv_vapic",
> the guests cannot boot.

Can we try without specifying +x2apic flag for Haswell/SandyBridge, they both
should have it be default.

Thanks,
Vadim.

(In reply to Vadim Rozenfeld from comment #4)
> > 2. on IvyBridge host, only with "-cpu Haswell/SandyBridge,+x2apic,hv_vapic",
> > the guests cannot boot.
> 
> Can we try without specifying +x2apic flag for Haswell/SandyBridge, they both
> should have it be default.
> 
> Thanks,
> Vadim.

Hi, Vadim,

On ivybridge host, I use "-cpu Haswell,hv_vapic" to boot a win2012r2 guest. It is also failed and the sceenshot is also as comment 1. The ivybridge host supports apicv:
# cat /sys/module/kvm_intel/parameters/enable_apicv
Y

Disable apicv on this host as following commands, use "-cpu Haswell,hv_vapic" to boot a win2012r2 guest, the guest can boot successfully.

# modprobe -r kvm_intel
# modprobe kvm_intel enable_apicv=N
# cat /sys/module/kvm_intel/parameters/enable_apicv
N


Thanks.
Huiqing

I think I understand what's going on here.

When Hyper-V enlightenments are in effect, Windows prefers to issue an Hyper-V 
MSR write to issue an EOI rather than an x2apic MSR write.  The Hyper-V MSR 
write is not handled by the processor, and on EOI the processor will modify the 
highest in-service interrupt (SVI) field of the VMCS, as explained in section 
29.1.4 of the SDM.  We need to do the same on Hyper-V MSR writes.

Can I have a test environment where I can test a fix?

> Can I have a test environment where I can test a fix?

Hi Huding,

Can we provide the host which can be stably reproduced this issue?

Best Regards,
Junyi

huiqingding, where can I find the Windows 2012 ISO?  Could you copy it for me to /root?  Thanks!

Created attachment 897538 [details]
tested patch

Brew build id 7474234

Reproduced this bug with <kernel 3.10.0-143.el7.x86_64.

result:
win2k8r2 guest black screen.

verified this bug with 3.10.0-143.el7.x86_64.

guest works well.

*** Bug 1129225 has been marked as a duplicate of this bug. ***

Test this bug on an Ivybridge host using the following version:
kernel-3.10.0-146.el7.x86_64
qemu-kvm-rhev-2.1.0-2.el7.x86_64

The host supports apicv:
# cat /sys/module/kvm_intel/parameters/enable_apicv
Y

Steps to test:
1. boot a win2k8r2 guest with "+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time"
# /usr/libexec/qemu-kvm -M pc -cpu Haswell,+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -name rhel7 -rtc base=localtime,clock=host,driftfix=slew  -device virtio-serial-pci,id=virtio-serial0 -chardev socket,id=channel1,path=/tmp/test1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.com,bus=virtio-serial0.0,id=port1  -drive file=/home/win2k8r2.raw,if=none,id=drive-virtio-disk0,format=raw,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -serial unix:/tmp/ttyS0,server,nowait -spice port=5900,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -monitor stdio -qmp tcp:0:4445,server,nowait -nodefaults -boot menu=on -cdrom 7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso -drive file=/home/driver.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,unit=1,ver=fuxc-ver-cdrom,bus=ide.0,unit=1

Actual results:
after step1, the guest can boot normally.

Moving to correct component

Reproduce this bug using the following version:
kernel-3.10.0-142.el7.x86_64
qemu-kvm-1.5.3-77.el7.x86_64

The host supports apicv:
# cat /sys/module/kvm_intel/parameters/enable_apicv
Y

Steps to Reproduce:
1. boot a win2012r2 guest with "+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time"
# /usr/libexec/qemu-kvm -M pc -cpu Haswell,+x2apic,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_time -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -name rhel7 -rtc base=localtime,clock=host,driftfix=slew  -device virtio-serial-pci,id=virtio-serial0 -chardev socket,id=channel1,path=/tmp/test1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.com,bus=virtio-serial0.0,id=port1  -drive file=/home/win2012r2-blk.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -serial unix:/tmp/ttyS0,server,nowait -spice port=5900,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -monitor stdio -qmp tcp:0:4445,server,nowait -nodefaults -boot menu=on -cdrom 9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_EN-US-IR3_SSS_X64FREE_EN-US_DV9.ISO -drive file=/home/driver.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-0,id=ide0-1-0,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,unit=1,ver=fuxc-ver-cdrom,bus=ide.0,unit=1

Actual results:
after step1, the guest cannot boot sucessfully.

Test this issue using the folliwng version:
kernel-3.10.0-196.el7.x86_64
qemu-kvm-1.5.3-77.el7.x86_64

Use the same command line of comment 26 to boot a win2012r2 guest and the result is ok: the guest can boot normally.

Test this issue using the folliwng version:
kernel-3.10.0-196.el7.x86_64
qemu-kvm-rhev-2.1.2-6.el7.x86_64

Use the same command line of comment 26 to boot a win2012r2 guest and the result is ok: the guest can boot normally.

According to comment26-comment28, set this issue as verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0290.html