Bug 804923

Summary: winxp BSOD happened during s3 due to memory corruption on AMD host
Product: Red Hat Enterprise Linux 6 Reporter: Mike Cao <bcao>
Component: qemu-kvmAssignee: Vadim Rozenfeld <vrozenfe>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.3CC: acathrow, amit.shah, areis, bcao, bsarathy, gleb, juzhang, michen, mkenneth, rhod, shuang, tburke, virt-maint, vrozenfe
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-05-21 17:46:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 761491, 912287    

Description Mike Cao 2012-03-20 08:08:57 UTC 
Description of problem:


Version-Release number of selected component (if applicable):
# uname -r
2.6.32-252.el6.x86_64
[root@amd-2376-32-1 /]#  rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.248.el6rhev.x86_64
seabios -13 


How reproducible:
100%

Steps to Reproduce:
1.start winxp guest 
/usr/libexec/qemu-kvm -M rhel6.3.0 -enable-kvm -m 4G -smp 4,sockets=4,cores=1,threads=1 -name winxp -uuid e2eaca3e-e764-f57b-22f0-74f4ab8c4965 -monitor stdio -rtc base=localtime,driftfix=slew -drive file=/test/winxp,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/root/en_windows_xp_professional_with_service_pack_3_x86_cd_x14-80428.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:15:af:6a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0  -spice port=5910,disable-ticketing -vga qxl -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -balloon none
2.s3 guest 
3.use mouse wake it up 
  
Actual results:
Guest BOSD 

Expected results:
Guest resume successfully.

Additional info:
Comment 1 Mike Cao 2012-03-20 08:10:20 UTC 
1: kd> !analyze -v 
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

ATTEMPTED_SWITCH_FROM_DPC (b8)
A wait operation, attach process, or yield was attempted from a DPC routine.
This is an illegal operation and the stack track will lead to the offending
code and original DPC routine.
Arguments:
Arg1: 00000000, Original thread which is the cause of the failure
Arg2: 00000000, New thread
Arg3: 00000000, Stack address of the original thread
Arg4: 00000000

Debugging Details:
------------------

PEB is paged out (Peb.Ldr = 7ffdf00c).  Type ".hh dbgerr001" for details
PEB is paged out (Peb.Ldr = 7ffdf00c).  Type ".hh dbgerr001" for details

DEFAULT_BUCKET_ID:  CODE_CORRUPTION

BUGCHECK_STR:  0xB8

PROCESS_NAME:  csrss.exe

LAST_CONTROL_TRANSFER:  from 80545c4b to 804f9f0e

STACK_TEXT:  
bacd7ee8 80545c4b 000000b8 ffffffff 00000202 nt!KeBugCheck+0x14
bacd7ef8 80545a57 bacd7f10 806546a3 00000041 nt!ScPatchFxe+0x46
bacd7f08 806e6d43 bacd7f90 806546a3 badb0d00 nt!KiDispatchInterrupt+0xa7
bacd7f08 806546a3 bacd7f90 806546a3 badb0d00 hal!HalpDispatchInterrupt+0xbb
bacd7f90 806547ab baa87594 bacd7fa0 00000005 nt!PopHandleNextState+0x1d
bacd7fcc 80545e6f baa87538 baa87594 00000000 nt!PopInvokeStateHandlerTargetProcessor+0x23
bacd7ff4 805459db b20d4d44 00000000 00000000 nt!KiRetireDpcList+0x61
bacd7ff8 b20d4d44 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x2b
WARNING: Frame IP not in any known module. Following frames may be wrong.
805459db 00000000 00000009 0081850f bb830000 0xb20d4d44


STACK_COMMAND:  kb

CHKIMG_EXTENSION: !chkimg -lo 50 -d !hal
    806e63c2-806e63c5  4 bytes - hal!KfRaiseIrql+a
	[ a1 80 00 fe:e8 f9 44 9e ]
    806e63c7-806e63cb  5 bytes - hal!KfRaiseIrql+f (+0x05)
	[ 89 0d 80 00 fe:51 e8 6b 45 9e ]
    806e63d8-806e63dc  5 bytes - hal!KeRaiseIrqlToDpcLevel (+0x11)
	[ 8b 15 80 00 fe:90 e8 25 45 9e ]
    806e63de-806e63e0  3 bytes - hal!KeRaiseIrqlToDpcLevel+6 (+0x06)
	[ c7 05 80:68 41 00 ]
    806e63e2-806e63e7  6 bytes - hal!KeRaiseIrqlToDpcLevel+a (+0x04)
	[ fe ff 41 00 00 00:00 e8 50 45 9e ff ]
    806e63f4-806e63f8  5 bytes - hal!KeRaiseIrqlToSynchLevel (+0x12)
	[ 8b 15 80 00 fe:90 e8 09 45 9e ]
    806e63fa-806e63fc  3 bytes - hal!KeRaiseIrqlToSynchLevel+6 (+0x06)
	[ c7 05 80:68 d1 00 ]
    806e63fe-806e6403  6 bytes - hal!KeRaiseIrqlToSynchLevel+a (+0x04)
	[ fe ff d1 00 00 00:00 e8 34 45 9e ff ]
    806e641c-806e6420  5 bytes - hal!KfLowerIrql+c (+0x1e)
	[ 89 0d 80 00 fe:51 e8 16 45 9e ]
    806e6422-806e6425  4 bytes - hal!KfLowerIrql+12 (+0x06)
	[ a1 80 00 fe:e8 99 44 9e ]
    806e6428-806e642b  4 bytes - hal!KeGetCurrentIrql (+0x06)
	[ a1 80 00 fe:e8 93 44 9e ]
    806e6830-806e6834  5 bytes - hal!KfAcquireSpinLock (+0x408)
	[ 8b 15 80 00 fe:90 e8 cd 40 9e ]
    806e6836-806e6838  3 bytes - hal!KfAcquireSpinLock+6 (+0x06)
	[ c7 05 80:68 41 00 ]
    806e683a-806e683f  6 bytes - hal!KfAcquireSpinLock+a (+0x04)
	[ fe ff 41 00 00 00:00 e8 f8 40 9e ff ]
    806e690f-806e6913  5 bytes - hal!KfReleaseSpinLock+f (+0xd5)
	[ 89 0d 80 00 fe:51 e8 23 40 9e ]
    806e6915-806e6918  4 bytes - hal!KfReleaseSpinLock+15 (+0x06)
	[ a1 80 00 fe:e8 a6 3f 9e ]
    806e6940-806e6943  4 bytes - hal!ExAcquireFastMutex (+0x2b)
	[ a1 80 00 fe:e8 7b 3f 9e ]
    806e6945-806e6947  3 bytes - hal!ExAcquireFastMutex+5 (+0x05)
	[ c7 05 80:68 3d 00 ]
    806e6949-806e694e  6 bytes - hal!ExAcquireFastMutex+9 (+0x04)
	[ fe ff 3d 00 00 00:00 e8 e9 3f 9e ff ]
    806e698f-806e6992  4 bytes - hal!ExReleaseFastMutex+1b (+0x46)
	[ a3 80 00 fe:e8 9d 3f 9e ]
    806e6994-806e6998  5 bytes - hal!ExReleaseFastMutex+20 (+0x05)
	[ 8b 0d 80 00 fe:90 e8 60 3f 9e ]
    806e69a1-806e69a4  4 bytes - hal!ExTryToAcquireFastMutex+5 (+0x0d)
	[ a1 80 00 fe:e8 1a 3f 9e ]
    806e69a6-806e69a8  3 bytes - hal!ExTryToAcquireFastMutex+a (+0x05)
	[ c7 05 80:68 3d 00 ]
    806e69aa-806e69af  6 bytes - hal!ExTryToAcquireFastMutex+e (+0x04)
	[ fe ff 3d 00 00 00:00 e8 88 3f 9e ff ]
    806e69e5-806e69e7  3 bytes - hal!KeAcquireInStackQueuedSpinLockRaiseToSynch+5 (+0x3b)
	[ c7 05 80:68 d1 00 ]
    806e69e9-806e69ee  6 bytes - hal!KeAcquireInStackQueuedSpinLockRaiseToSynch+9 (+0x04)
	[ fe ff d1 00 00 00:00 e8 49 3f 9e ff ]
    806e69f0-806e69f3  4 bytes - hal!KeAcquireInStackQueuedSpinLock (+0x07)
	[ a1 80 00 fe:e8 cb 3e 9e ]
    806e69fe-806e6a00  3 bytes - hal!KeAcquireInStackQueuedSpinLock+e (+0x0e)
	[ c7 05 80:68 41 00 ]
    806e6a02-806e6a07  6 bytes - hal!KeAcquireInStackQueuedSpinLock+12 (+0x04)
	[ fe ff 41 00 00 00:00 e8 30 3f 9e ff ]
    806e6a41-806e6a43  3 bytes - hal!KeAcquireQueuedSpinLockRaiseToSynch+5 (+0x3f)
	[ c7 05 80:68 d1 00 ]
    806e6a45-806e6a4a  6 bytes - hal!KeAcquireQueuedSpinLockRaiseToSynch+9 (+0x04)
	[ fe ff d1 00 00 00:00 e8 ed 3e 9e ff ]
    806e6a4c-806e6a4f  4 bytes - hal!KeAcquireQueuedSpinLock (+0x07)
	[ a1 80 00 fe:e8 6f 3e 9e ]
    806e6a5b-806e6a5d  3 bytes - hal!KeAcquireQueuedSpinLock+f (+0x0f)
	[ c7 05 80:68 41 00 ]
    806e6a5f-806e6a64  6 bytes - hal!KeAcquireQueuedSpinLock+13 (+0x04)
	[ fe ff 41 00 00 00:00 e8 d3 3e 9e ff ]
    806e6ad9-806e6add  5 bytes - hal!KeReleaseQueuedSpinLock+31 (+0x7a)
	[ 89 0d 80 00 fe:51 e8 59 3e 9e ]
    806e6adf-806e6ae2  4 bytes - hal!KeReleaseQueuedSpinLock+37 (+0x06)
	[ a1 80 00 fe:e8 dc 3d 9e ]
    806e6b39-806e6b3d  5 bytes - hal!KeTryToAcquireQueuedSpinLock+31 (+0x5a)
	[ 8b 0d 80 00 fe:90 e8 bb 3d 9e ]
    806e6b3f-806e6b42  4 bytes - hal!KeTryToAcquireQueuedSpinLock+37 (+0x06)
	[ a3 80 00 fe:e8 ed 3d 9e ]
    806e6d27-806e6d2b  5 bytes - hal!HalpDispatchInterrupt+9f (+0x1e8)
	[ 8b 35 80 00 fe:90 e8 df 3b 9e ]
    806e6d2d-806e6d30  4 bytes - hal!HalpDispatchInterrupt+a5 (+0x06)
	[ a3 80 00 fe:e8 ff 3b 9e ]
    806e6d44-806e6d48  5 bytes - hal!HalpDispatchInterrupt+bc (+0x17)
	[ 89 35 80 00 fe:56 e8 ee 3b 9e ]
    806e6d4a-806e6d4e  5 bytes - hal!HalpDispatchInterrupt+c2 (+0x06)
	[ 8b 0d 80 00 fe:90 e8 aa 3b 9e ]
    806e6ecb-806e6ecf  5 bytes - hal!HalpApcInterrupt+9f (+0x181)
	[ 8b 0d 80 00 fe:90 e8 29 3a 9e ]
    806e6ed2-806e6ed5  4 bytes - hal!HalpApcInterrupt+a6 (+0x07)
	[ a3 80 00 fe:e8 5a 3a 9e ]
    806e6ef4-806e6ef7  4 bytes - hal!HalpApcInterrupt+c8 (+0x22)
	[ a3 80 00 fe:e8 38 3a 9e ]
    806e6ef9-806e6efd  5 bytes - hal!HalpApcInterrupt+cd (+0x05)
	[ 8b 0d 80 00 fe:90 e8 fb 39 9e ]
    806e6f27-806e6f2b  5 bytes - hal!HalEndSystemInterrupt+1b (+0x2e)
	[ 89 0d 80 00 fe:51 e8 0b 3a 9e ]
    806e6f2d-806e6f31  5 bytes - hal!HalEndSystemInterrupt+21 (+0x06)
	[ 8b 15 80 00 fe:90 e8 d0 39 9e ]
    806e6f48-806e6f4a  3 bytes - hal!HalEndSystemInterrupt+3c (+0x1b)
	[ c7 05 80:68 41 00 ]
    806e6f4c-806e6f51  6 bytes - hal!HalEndSystemInterrupt+40 (+0x04)
	[ fe ff 41 00 00 00:00 e8 e6 39 9e ff ]
WARNING: !chkimg output was truncated to 50 lines. Invoke !chkimg without '-lo [num_lines]' to view  entire output.
    806fa0ba-806fa0bc  3 bytes - hal!HalpInitializeLocalUnit+86
	[ c7 05 80:68 ff 00 ]
    806fa0be-806fa0c3  6 bytes - hal!HalpInitializeLocalUnit+8a (+0x04)
	[ fe ff ff 00 00 00:00 e8 74 08 9d ff ]
    806fa18d-806fa191  5 bytes - hal!HalpInitializeLocalUnit+159 (+0xcf)
	[ 89 1d 80 00 fe:53 e8 a5 07 9d ]
248 errors : !hal (806e63c2-806fa191)

MODULE_NAME: memory_corruption

IMAGE_NAME:  memory_corruption

FOLLOWUP_NAME:  memory_corruption

DEBUG_FLR_IMAGE_TIMESTAMP:  0

MEMORY_CORRUPTOR:  LARGE

FAILURE_BUCKET_ID:  MEMORY_CORRUPTION_LARGE

BUCKET_ID:  MEMORY_CORRUPTION_LARGE

Followup: memory_corruption
---------
Comment 4 Mike Cao 2012-03-20 09:34:47 UTC 
BTW  ,I tried win2k8R2 ,did not hit this issue .

BOSD can not 100% reproduce , when guest did not get BSOD ,it will hang and never resume from s3
Comment 9 Mike Cao 2012-03-23 08:09:55 UTC 
This Bug may related to hardware ,since what I use is a freshly installed one ,and I downgrade the qemu-kvm version ,still hit this issue .
Comment 10 Ademar Reis 2012-03-28 14:38:51 UTC 
(In reply to comment #9)
> This Bug may related to hardware ,since what I use is a freshly installed one
> ,and I downgrade the qemu-kvm version ,still hit this issue .

Do you mean a problem with the host hardware? Anyway, please reproduce the problem with a different machine then.
Comment 11 Mike Cao 2012-03-29 03:02:49 UTC 
(In reply to comment #10)
> (In reply to comment #9)
> > This Bug may related to hardware ,since what I use is a freshly installed one
> > ,and I downgrade the qemu-kvm version ,still hit this issue .
> 
> Do you mean a problem with the host hardware? Anyway, please reproduce the
> problem with a different machine then.

Hello Ademar

Another QE can *not* reproduce on her host with the same winxp image .but it happened almost 100% in amd-2376-32-1 host .
And on amd-2376-32-1 host,I tried w/ win2k8R2 guests ,can *not* reproduce this issue 

Based on above ,I think this bug might related to specific host(hardware)

Best Regards,
Mike
Comment 12 Ademar Reis 2012-03-29 15:43:37 UTC 
(In reply to comment #11)
> (In reply to comment #10)
> > (In reply to comment #9)
> > > This Bug may related to hardware ,since what I use is a freshly installed one
> > > ,and I downgrade the qemu-kvm version ,still hit this issue .
> > 
> > Do you mean a problem with the host hardware? Anyway, please reproduce the
> > problem with a different machine then.
> 
> Hello Ademar
> 
> Another QE can *not* reproduce on her host with the same winxp image .but it
> happened almost 100% in amd-2376-32-1 host .
> And on amd-2376-32-1 host,I tried w/ win2k8R2 guests ,can *not* reproduce this
> issue 
> 
> Based on above ,I think this bug might related to specific host(hardware)

Hmm, ok, let's try to narrow it a bit more: can you reproduce it on a similar hardware, on a different machine? When you think it might be related to specific hardware, do you mean some specific kind of hardware or your machine which might be deffective?

I assume amd-2376-32-1 is a machine we can access remotely, is that right?

Thanks!
Comment 14 Mike Cao 2012-03-30 09:28:55 UTC 
(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #10)
> > > (In reply to comment #9)
> > > > This Bug may related to hardware ,since what I use is a freshly installed one
> > > > ,and I downgrade the qemu-kvm version ,still hit this issue .
> > > 
> > > Do you mean a problem with the host hardware? Anyway, please reproduce the
> > > problem with a different machine then.
> > 
> > Hello Ademar
> > 
> > Another QE can *not* reproduce on her host with the same winxp image .but it
> > happened almost 100% in amd-2376-32-1 host .
> > And on amd-2376-32-1 host,I tried w/ win2k8R2 guests ,can *not* reproduce this
> > issue 
> > 
> > Based on above ,I think this bug might related to specific host(hardware)
> 
> Hmm, ok, let's try to narrow it a bit more: can you reproduce it on a similar
> hardware, on a different machine? When you think it might be related to
> specific hardware, do you mean some specific kind of hardware or your machine
> which might be deffective?

Hello Ademar

This bug related w/ host cpu , I lent another AMD host ,very easy to reproduce it 

Best Regards,
Mike
Comment 17 RHEL Program Management 2012-07-10 07:50:49 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 18 RHEL Program Management 2012-07-11 02:03:34 UTC 
This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development.  This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.
Comment 21 Ronen Hod 2013-05-21 17:46:48 UTC 
Closing
WinXP. Not a customer bug. Not sure that reproducible.