Description of problem: BSOD when execute sandra muti-media benchmark Version-Release number of selected component (if applicable): kvm-83-155.el5 How reproducible: 100% Steps to Reproduce: 1. Boot guest /usr/libexec/qemu-kvm -smp 2 -m 2G -drive file=/root/win08-r2.bak -net nic,vlan=0,macaddr=00:20:10:21:00:37,model=virtio -net tap,vlan=0,script=/etc/qemu-ifup -uuid `uuidgen` -no-hpet -usbdevice tablet -rtc-td-hack -startdate now -cpu qemu64,+sse2,+vmx,+ssse3,+popcnt,+sse4.1,+sse4.2 -monitor stdio -boot c -vnc :6 2. run sandra muti-media benchmark, you can download here http://download.cnet.com/SiSoftware-Sandra/3000-2086_4-10556571.html 3. Actual results: Expected results: Additional info: 1. Bugcheck Analysis : SYSTEM_SERVICE_EXCEPTION (3b) An exception happened while executing a system service routine. Arguments: Arg1: 00000000c0000096, Exception code that caused the bugcheck Arg2: fffff88003bbe6bb, Address of the exception record for the exception that caused the bugcheck Arg3: fffff880040fdfc0, Address of the context record for the exception that caused the bugcheck 2. when boot guest with -cpu qemu64 don't cause BSOD. 3. host processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm 4. guest windows08-R2-64
vmx lead to BSOD
(In reply to comment #1) > vmx lead to BSOD Why not analyze the BSOD?! Please see http://cleo.tlv.redhat.com/qumrawiki/QA/KB/!WindowsCrash
Created attachment 388463 [details] BSOD dump I paste "Bugcheck Analysis" on Additional info above, details please refer to the attachment.
(In reply to comment #3) > Created an attachment (id=388463) [details] > BSOD dump > > I paste "Bugcheck Analysis" on Additional info above, details please refer to > the attachment. Why not post the complete bug analysis? '!analyze -v' would give you a lot more information.
(In reply to comment #2) > > Please see http://cleo.tlv.redhat.com/qumrawiki/QA/KB/!WindowsCrash At face value it appears the crash is a result of a privilege violation attempting to execute an 'rdmsr' instruction, which seems a little odd relative to the intel doc so I'd hazard it is rather a general protection exception. If all that holds it appears the most likely cause is a bad msr address in ecx (0x480). Maybe the given mix of cpu flags is confusing the test as to what machine actually exists. Can you determine if the fault corresponds to one flag (+sse2,+vmx,+ssse3,+popcnt,+sse4.1,+sse4.2) in particular?
vmx cause BSOD
(In reply to comment #6) > vmx cause BSOD Backing up a bit here, do we know for a fact the Sandra benchmark is known to work with vmx enabled? If so: Is there a way to extract a guest dump of the entire CPUID state when vmx is enabled and the benchmark executes normally? My guess (in the case vmx is needed or tolerated) is the benchmark stumbling upon seeing vmx enabled when some other expected guest feature is unintentionally missing.
Run sandra on guests: 1. guest windows-xp BSOD every time with any -cpu option. 2. guest windows-08-R2 BSOD every time when boot with vmx (five times) 3. guest windows-08-R2 doen't BSOD when boot without vmx x86info with vmx and without vmx 1). -cpu qemu64,+sse2,+ssse3 -------------------------------------------------------------------------- CPU #2 eax in: 0x00000000, eax = 00000004 ebx = 756e6547 ecx = 6c65746e edx = 49656e eax in: 0x00000001, eax = 00000663 ebx = 00000800 ecx = 80000201 edx = 078bfb eax in: 0x00000002, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 002c30 eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x00000004, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x80000000, eax = 8000000a ebx = 68747541 ecx = 444d4163 edx = 69746e eax in: 0x80000001, eax = 078bfbfd ebx = 00000000 ecx = 00000000 edx = 2191ab eax in: 0x80000002, eax = 554d4551 ebx = 72695620 ecx = 6c617574 edx = 555043 eax in: 0x80000003, eax = 72657620 ebx = 6e6f6973 ecx = 392e3020 edx = 000031 eax in: 0x80000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x80000005, eax = 01ff01ff ebx = 01ff01ff ecx = 40020140 edx = 400201 eax in: 0x80000006, eax = 00000000 ebx = 42004200 ecx = 02008140 edx = 000000 eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x80000008, eax = 00003028 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000000 eax in: 0x8000000a, eax = 00000001 ebx = 00000010 ecx = 00000000 edx = 000000 Family: 6 Model: 6 Stepping: 3 Type: 0 Brand: 0 CPU Model: Celeron / Mobile Pentium II Original OEM Feature flags: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh m fxsr sse sse2 Extended feature flags: sse3 ssse3 [31] [0] [2] [3] [4] [5] [6] [7] [8] [9] SYSCALL [13] [15] [16] xd [23] [24] em64 Cache info L1 Instruction cache: 32KB, 8-way associative. 64 byte line size. L1 Data cache: 32KB, 8-way associative. 64 byte line size. L2 unified cache: 2MB, sectored, 8-way associative. 64 byte line size. TLB info Connector type: Socket 370 (370 Pin PGA) MTRR registers: MTRRcap (0xfe): MTRRphysBase0 (0x200): MTRRphysMask0 (0x201): MTRRphysBase1 ( 02): MTRRphysMask1 (0x203): MTRRphysBase2 (0x204): MTRRphysMask2 (0x205): MTR ysBase3 (0x206): MTRRphysMask3 (0x207): MTRRphysBase4 (0x208): MTRRphysMask4 209): MTRRphysBase5 (0x20a): MTRRphysMask5 (0x20b): MTRRphysBase6 (0x20c): MT hysMask6 (0x20d): MTRRphysBase7 (0x20e): MTRRphysMask7 (0x20f): MTRRfix64K_00 (0x250): MTRRfix16K_80000 (0x258): MTRRfix16K_A0000 (0x259): MTRRfix4K_C8000 x269): MTRRfix4K_D0000 0x26a: MTRRfix4K_D8000 0x26b: MTRRfix4K_E0000 0x26c: M fix4K_E8000 0x26d: MTRRfix4K_F0000 0x26e: MTRRfix4K_F8000 0x26f: MTRRdefType 2ff): 2.65GHz processor (estimate). 2). -cpu qemu64,+sse2,+ssse3,+vmx CPU #2 eax in: 0x00000000, eax = 00000004 ebx = 756e6547 ecx = 6c65746e edx = 49656e69 eax in: 0x00000001, eax = 00000663 ebx = 00000800 ecx = 80000221 edx = 078bfbfd eax in: 0x00000002, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 002c307d eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x00000004, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000000, eax = 8000000a ebx = 68747541 ecx = 444d4163 edx = 69746e65 eax in: 0x80000001, eax = 078bfbfd ebx = 00000000 ecx = 00000000 edx = 2191abfd eax in: 0x80000002, eax = 554d4551 ebx = 72695620 ecx = 6c617574 edx = 55504320 eax in: 0x80000003, eax = 72657620 ebx = 6e6f6973 ecx = 392e3020 edx = 0000312e eax in: 0x80000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000005, eax = 01ff01ff ebx = 01ff01ff ecx = 40020140 edx = 40020140 eax in: 0x80000006, eax = 00000000 ebx = 42004200 ecx = 02008140 edx = 00000000 eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000008, eax = 00003028 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x8000000a, eax = 00000001 ebx = 00000010 ecx = 00000000 edx = 00000000 Family: 6 Model: 6 Stepping: 3 Type: 0 Brand: 0 CPU Model: Celeron / Mobile Pentium II Original OEM Feature flags: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 Extended feature flags: sse3 vmx ssse3 [31] [0] [2] [3] [4] [5] [6] [7] [8] [9] SYSCALL [13] [15] [16] xd [23] [24] em64t Cache info L1 Instruction cache: 32KB, 8-way associative. 64 byte line size. L1 Data cache: 32KB, 8-way associative. 64 byte line size. L2 unified cache: 2MB, sectored, 8-way associative. 64 byte line size. TLB info Connector type: Socket 370 (370 Pin PGA) MTRR registers: MTRRcap (0xfe): MTRRphysBase0 (0x200): MTRRphysMask0 (0x201): MTRRphysBase1 (0x2 02): MTRRphysMask1 (0x203): MTRRphysBase2 (0x204): MTRRphysMask2 (0x205): MTRRph ysBase3 (0x206): MTRRphysMask3 (0x207): MTRRphysBase4 (0x208): MTRRphysMask4 (0x 209): MTRRphysBase5 (0x20a): MTRRphysMask5 (0x20b): MTRRphysBase6 (0x20c): MTRRp hysMask6 (0x20d): MTRRphysBase7 (0x20e): MTRRphysMask7 (0x20f): MTRRfix64K_00000 (0x250): MTRRfix16K_80000 (0x258): MTRRfix16K_A0000 (0x259): MTRRfix4K_C8000 (0 x269): MTRRfix4K_D0000 0x26a: MTRRfix4K_D8000 0x26b: MTRRfix4K_E0000 0x26c: MTRR fix4K_E8000 0x26d: MTRRfix4K_F0000 0x26e: MTRRfix4K_F8000 0x26f: MTRRdefType (0x 2ff): 2.65GHz processor (estimate).
It looks like the above dump was truncated: (In reply to comment #8) > Run sandra on guests: > > 1. guest windows-xp BSOD every time with any -cpu option. > 2. guest windows-08-R2 BSOD every time when boot with vmx (five times) > 3. guest windows-08-R2 doen't BSOD when boot without vmx I see the corresponding command line flags for case #1 & #2 but not for case #3. > eax in: 0x00000000, eax = 00000004 ebx = 756e6547 ecx = 6c65746e edx = 49656e > eax in: 0x00000001, eax = 00000663 ebx = 00000800 ecx = 80000201 edx = 078bfb All of this dump is seems to be truncated horizontally at 77 characters. Could you re-attach the log in full?
Created attachment 399526 [details] x86info when boot windows08_R2 wich -cpu qemu64,+sse2,+ssse3
Created attachment 399527 [details] x86info when boot windows08_R2 wich -cpu qemu64,+sse2,+ssse3,+vmx
Created attachment 399528 [details] x86info of windows-xp when boot with -cpu qemu64,+sse2,+ssse3
Just noticed this may be another symptom similar to that fixed in BZ #636494. +vmx may just be confusing the guest and needs to be trivially filtered from the allowable flag set.
(In reply to comment #14) > Just noticed this may be another symptom similar to that fixed in > BZ #636494. +vmx may just be confusing the guest and needs to be > trivially filtered from the allowable flag set. The guest appears to be accessing MSR_IA32_VMX_BASIC in the presence of vmx. This msr isn't handled by kvm, although it isn't clear how access of this is related to sandra running. Pass of vmx should be disabled here assuming libvirt will pass +vmx in some scenario. It isn't obvious yet [to me] this is possible.
While Jiri Denemark will be a more definitive answer, my understanding is that libvirt uses a guest's XML request for particular cpu flags in the <cpu> element (such as <feature policy='require' name='vmx'/>) to drive the qemu command line. If the <model> element was present, then libvirt guarantees that the combination is supported by that model, and the RHEL 5.6 cpu_map.xml does not list the 'vmx' flags in any of the base models. See http://libvirt.org/formatdomain.html#elementsCPU for more on what libvirt XML will permit. But I'm afraid that without a libvirt patch, it might indeed be possible to create XML in a RHEL 5 host that will cause libvirt to request the +vmx flag from qemu.
Created attachment 490128 [details] Proposed patch to resolve case.
Created attachment 490794 [details] Proposed patch to resolve case.
Libvirt's action depends on what a user specifies in domain XML. Since no CPU model currently has vmx flag included, it's pretty easy to say when libvirt passes explicit +vmx to qemu: it does so iff vmx CPU feature is explicitly requested in <cpu> element (e.g., by <feature policy='require' name='vmx'/>).
Fixed in kvm-83-232.el5 cmd: /usr/libexec/qemu-kvm -drive file='/home/images/win2008r2-64-virtio.qcow2',index=0,if=virtio,media=disk,cache=none,boot=on,format=qcow2 -net nic,vlan=0,model=virtio,macaddr='9a:5c:aa:e0:9c:81' -net tap,vlan=0,script='/home/scripts/qemu-ifup',downscript='no' -m 8192 -smp 4,cores=2,threads=1,sockets=2 -cpu qemu64,+sse2,+sse4.1,+sse4.2,+vmx -soundhw ac97 -redir tcp:5000::10023 -vnc :0 -rtc-td-hack -M rhel5.6.0 -boot c -usbdevice tablet -monitor stdio 1. reproduce in kvm-83-224.el5 1). x86info : (vmx is exported to guest) CPU #4 eax in: 0x00000000, eax = 00000004 ebx = 756e6547 ecx = 6c65746e edx = 49656e69 eax in: 0x00000001, eax = 00000663 ebx = 01020800 ecx = 80180021 edx = 178bfbfd eax in: 0x00000002, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 002c307d eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x00000004, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000000, eax = 8000000a ebx = 68747541 ecx = 444d4163 edx = 69746e65 eax in: 0x80000001, eax = 078bfbfd ebx = 00000000 ecx = 00000002 edx = 2191abfd eax in: 0x80000002, eax = 554d4551 ebx = 72695620 ecx = 6c617574 edx = 55504320 eax in: 0x80000003, eax = 72657620 ebx = 6e6f6973 ecx = 392e3020 edx = 0000312e eax in: 0x80000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000005, eax = 01ff01ff ebx = 01ff01ff ecx = 40020140 edx = 40020140 eax in: 0x80000006, eax = 00000000 ebx = 42004200 ecx = 02008140 edx = 00000000 eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000008, eax = 00003028 ebx = 00000000 ecx = 00000001 edx = 00000000 eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x8000000a, eax = 00000001 ebx = 00000010 ecx = 00000000 edx = 00000000 Family: 6 Model: 6 Stepping: 3 Type: 0 Brand: 0 CPU Model: Celeron / Mobile Pentium II Original OEM Feature flags: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht Extended feature flags: sse3 vmx [19] [20] [31] [0] [2] [3] [4] [5] [6] [7] [8] [9] SYSCALL [13] [15] [16] xd [23] [24] em64t [1] Cache info L1 Instruction cache: 32KB, 8-way associative. 64 byte line size. L1 Data cache: 32KB, 8-way associative. 64 byte line size. L2 unified cache: 2MB, sectored, 8-way associative. 64 byte line size. TLB info The physical package supports 2 logical processors Connector type: Socket 370 (370 Pin PGA) 2). run sandra in guest guest BSOD 2. Fixed in kvm-83-232.el5 1). vmx is not exported to guest CPU #4 eax in: 0x00000000, eax = 00000004 ebx = 756e6547 ecx = 6c65746e edx = 49656e69 eax in: 0x00000001, eax = 00000663 ebx = 02020800 ecx = 80180001 edx = 178bfbfd eax in: 0x00000002, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 002c307d eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x00000004, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000000, eax = 8000000a ebx = 68747541 ecx = 444d4163 edx = 69746e65 eax in: 0x80000001, eax = 078bfbfd ebx = 00000000 ecx = 00000002 edx = 2191abfd eax in: 0x80000002, eax = 554d4551 ebx = 72695620 ecx = 6c617574 edx = 55504320 eax in: 0x80000003, eax = 72657620 ebx = 6e6f6973 ecx = 392e3020 edx = 0000312e eax in: 0x80000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000005, eax = 01ff01ff ebx = 01ff01ff ecx = 40020140 edx = 40020140 eax in: 0x80000006, eax = 00000000 ebx = 42004200 ecx = 02008140 edx = 00000000 eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x80000008, eax = 00003028 ebx = 00000000 ecx = 00000001 edx = 00000000 eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000 eax in: 0x8000000a, eax = 00000001 ebx = 00000010 ecx = 00000000 edx = 00000000 Family: 6 Model: 6 Stepping: 3 Type: 0 Brand: 0 CPU Model: Celeron / Mobile Pentium II Original OEM Feature flags: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht Extended feature flags: sse3 [19] [20] [31] [0] [2] [3] [4] [5] [6] [7] [8] [9] SYSCALL [13] [15] [16] xd [23] [24] em64t [1] Cache info L1 Instruction cache: 32KB, 8-way associative. 64 byte line size. L1 Data cache: 32KB, 8-way associative. 64 byte line size. L2 unified cache: 2MB, sectored, 8-way associative. 64 byte line size. TLB info The physical package supports 2 logical processors 2). can run sandra muti-media benchmark successfully 3. guest win2008r2 4. host processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm bogomips : 5319.96 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: When the Sandra multi-media benchmark utility was run on a Windows guest, the guest terminated unexpectedly when the utility tried to access the Model Specific Register 0x480 (IA32_VMX_BASIC). A patch has been provided to address this issue and the benchmark utility no longer causes a Windows guest to crash.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2011-1068.html