Bug 614722 - [WHQL] Win2k8-R2 got BSOD during virtio-serial run simpleIOStress test (Error code 0x1E)
Summary: [WHQL] Win2k8-R2 got BSOD during virtio-serial run simpleIOStress test (Error...
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: virtio-win   
(Show other bugs)
Version: 6.0
Hardware: All Linux
high
high
Target Milestone: rc
: ---
Assignee: Vadim Rozenfeld
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Keywords: TestBlocker
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-07-15 05:28 UTC by Qunfang Zhang
Modified: 2013-01-09 22:52 UTC (History)
9 users (show)

Fixed In Version: 1.1.12-0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-11-11 15:01:46 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
BSOD screenshot (21.26 KB, image/png)
2010-07-15 05:29 UTC, Qunfang Zhang 		no flags Details
Add an attachment (proposed patch, testcase, etc.)

Description Qunfang Zhang 2010-07-15 05:28:48 UTC 
Description of problem:
As subject, the BSOD happened when did virtio-serial test, and the job is  "DPWLK-Hot-Add-Device Test-Verify driver support for Hot-Add CPU", and the test case is "Run SimpleIOStress". Screenshot will be attached.

Version-Release number of selected component (if applicable):
virtio-win-1.1.7-2
qemu-kvm-0.12.1.2-2.91.el6.x86_64
2.6.32-44.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Boot a win2k8-R2 guest
/usr/libexec/qemu-kvm -m 6G -smp 4 -cpu qemu64,+x2apic -usbdevice tablet -drive file=win2k8-R2-serial.qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none,serial=win2k8-r2-229-last -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=00:10:1a:4a:21:1f,bus=pci.0,addr=0x4,id=net0 -boot c -uuid 911cf188-cd4e-486b-840d-81e126448e41 -rtc-td-hack -no-kvm-pit-reinjection -monitor stdio -name win2k8-R2-serial-229-last -vnc :11 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=4,bus=pci.0 -chardev pty,id=channel0 -device virtserialport,chardev=channel0,name=org.linux-kvm.port.0,bus=virtio-serial0.0

2.Install virtio-serial driver and prepare the environment.
3.Run job "DPWLK-Hot-Add-Device Test-Verify driver support for Hot-Add CPU"
  
Actual results:
Guest got BSOD.

Expected results:
Test passed.

Additional info:

Analyze the dump file:

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

KMODE_EXCEPTION_NOT_HANDLED (1e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: 0000000000000000, The exception code that was not handled
Arg2: 0000000000000000, The address that the exception occurred at
Arg3: 0000000000000000, Parameter 0 of the exception
Arg4: 0000000000000000, Parameter 1 of the exception

Debugging Details:
------------------

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
***                                                                   ***
***                                                                   ***
***    Your debugger is not using the correct symbols                 ***
***                                                                   ***
***    In order for this command to work properly, your symbol path   ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************



MODULE_NAME: vioser

FAULTING_MODULE: fffff80001465000 nt

DEBUG_FLR_IMAGE_TIMESTAMP:  4c2e1e75

EXCEPTION_CODE: (Win32) 0 (0) - The operation completed successfully.

FAULTING_IP: 
+0
00000000`00000000 ??              ???

EXCEPTION_PARAMETER1:  0000000000000000

EXCEPTION_PARAMETER2:  0000000000000000

DEFAULT_BUCKET_ID:  WRONG_SYMBOLS

BUGCHECK_STR:  0x1E

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from fffff800014cd32e to fffff800014d55d0

STACK_TEXT:  
fffff880`0306dc58 fffff800`014cd32e : 00000000`00000000 fffffa80`06089b60 fffff880`0306e3d0 fffff800`01502524 : nt!KeBugCheck
fffff880`0306dc60 fffff800`014fb2ed : fffff800`016e3b88 fffff800`0161ca40 fffff800`01465000 fffff880`0306eb68 : nt!KiCpuId+0x41e
fffff880`0306dc90 fffff800`01502950 : fffff800`01624a50 fffff880`0306dd08 fffff880`0306eb68 fffff800`01465000 : nt!KeReleaseQueuedSpinLock+0xdd
fffff880`0306dcc0 fffff800`0150f8df : fffff880`0306eb68 fffff880`0306e3d0 fffff880`00000000 fffffa80`05aa8348 : nt!FsRtlLookupLastBaseMcbEntry+0x4d0
fffff880`0306e3a0 fffff800`014d4c42 : fffff880`0306eb68 fffffa80`05b0f000 fffff880`0306ec10 fffffa80`05aa8348 : nt!FsRtlInitializeBaseMcbEx+0x430b
fffff880`0306ea30 fffff800`014d354a : fffff880`011798d0 00000000`00000058 fffff880`0306ed00 fffff880`0112e2dc : nt!KeSynchronizeExecution+0x3e32
fffff880`0306ec10 fffff880`029765d0 : 00000000`00000000 fffffa80`05acb240 fffffa80`062a0630 fffffa80`05417390 : nt!KeSynchronizeExecution+0x273a
fffff880`0306eda0 fffff880`02976a7a : fffffa80`05acb240 fffffa80`05af4000 0000057f`fabe8fd8 fffffa80`05aa8348 : vioser+0x25d0
fffff880`0306edd0 fffff880`02975889 : fffffa80`062a0630 00000000`00000000 fffffa80`05b25000 00000000`00000002 : vioser+0x2a7a
fffff880`0306ee20 fffff880`011668a7 : fffffa80`00000008 0000057f`fab77fd8 fffffa80`053f1d02 fffffa80`053f1d20 : vioser+0x1889
fffff880`0306eed0 fffff800`014e0cdc : fffff880`041ef180 fffffa80`053f1d20 fffffa80`053f1d38 00000000`0000ffff : Wdf01000+0x738a7
fffff880`0306ef00 fffff800`014db765 : 00000000`00000000 fffffa80`05d059e0 00000000`00000000 fffff880`01166818 : nt!KeRemoveQueueEx+0xe1c
fffff880`0306efb0 fffff800`014db57c : 00000000`000000a8 fffff880`02975777 fffffa80`05488020 00000000`00000000 : nt!SeAccessCheckWithHint+0xdd5
fffff880`023d2fb0 fffff800`01520b13 : fffff800`014d1436 fffff800`014d14a2 00000000`000000a8 00000000`00000001 : nt!SeAccessCheckWithHint+0xbec
fffff880`023d2fe0 fffff800`014d14a2 : 00000000`000000a8 00000000`00000001 fffffa80`06361d80 00000000`00000000 : nt!ExDisableResourceBoostLite+0x2c3
fffff880`023d2ff0 fffff800`01430b24 : 00000000`000000a8 fffff880`023d3300 00000000`00000001 fffff880`023d33f0 : nt!KeSynchronizeExecution+0x692
fffff880`023d3180 fffff800`01427155 : fffff880`023d33d0 fffff880`023d330a fffff880`023d33d0 00000000`00000001 : hal!HalGetProcessorIdByNtNumber+0x508
fffff880`023d31e0 fffff800`015934eb : 00000000`0000000b 00000000`00000001 fffff880`023d33d0 00000000`00000002 : hal!HalEnableInterrupt+0x79
fffff880`023d3230 fffff800`018d750b : 00000000`00000003 00000000`00000002 00000000`00000001 fffffa80`061c87a0 : nt!KeRevertToUserAffinityThreadEx+0xa0b
fffff880`023d3270 fffff800`0190c8b4 : fffff880`023d34e0 fffff880`023d34e0 fffffa80`053f2e70 fffffa80`05480000 : nt!IoDisconnectInterruptEx+0x3ab
fffff880`023d3360 fffff880`01166a82 : fffffa80`05e3f428 fffffa80`05488020 fffffa80`05488090 fffff880`023d35b0 : nt!IoConnectInterruptEx+0x224
fffff880`023d3440 fffff880`01162247 : fffffa80`05488020 00000000`00000000 fffffa80`053be3c0 fffffa80`05488180 : Wdf01000+0x73a82
fffff880`023d3540 fffff880`0115fa90 : fffffa80`053be020 00000000`00000000 fffff880`011745e0 fffffa80`053be020 : Wdf01000+0x6f247
fffff880`023d35a0 fffff880`0115f6eb : 00000000`00000000 00000000`00000000 fffff880`011745c0 00000000`00000000 : Wdf01000+0x6ca90
fffff880`023d3600 fffff880`0115f38e : fffffa80`053be020 00000000`00000040 fffff880`01174680 00000000`00000040 : Wdf01000+0x6c6eb
fffff880`023d3730 fffff880`0115f06c : 00000000`00000000 00000000`00000040 fffff880`01171da0 fffffa80`053be020 : Wdf01000+0x6c38e
fffff880`023d37a0 fffff880`01168c34 : 00000000`00000000 00000000`00000000 fffffa80`053be020 fffff880`01171da0 : Wdf01000+0x6c06c
fffff880`023d3830 fffff880`01168b37 : 00000000`00000000 00000000`00000000 fffff880`01171da0 fffff800`01664880 : Wdf01000+0x75c34
fffff880`023d3860 fffff880`01168691 : fffffa80`053be020 00000000`00000001 fffff880`011729e0 00000000`00000001 : Wdf01000+0x75b37
fffff880`023d3990 fffff880`01168372 : 00000000`00000000 00000000`00000001 00000000`00000118 00000000`00000118 : Wdf01000+0x75691
fffff880`023d3a00 fffff880`011654f8 : 00000000`00000000 00000000`00000118 00000000`00000118 00000000`00000118 : Wdf01000+0x75372
fffff880`023d3a90 fffff880`01164841 : 00000000`00000118 00000000`0000011d 00000000`0000011d 00000000`00000000 : Wdf01000+0x724f8
fffff880`023d3ac0 fffff880`011644fe : fffffa80`053be020 fffff880`023d3c20 00000000`00000004 fffff880`011736b0 : Wdf01000+0x71841
fffff880`023d3b30 fffff880`011643a4 : fffffa80`053be128 00000000`00000000 fffff880`023d3c20 fffffa80`053be020 : Wdf01000+0x714fe
fffff880`023d3ba0 fffff880`0116b59f : fffffa80`053be128 fffffa80`05d059e0 fffffa80`053be020 fffffa80`053be128 : Wdf01000+0x713a4
fffff880`023d3be0 fffff880`0116c8de : 00000000`00000000 fffffa80`053ef450 fffff800`0167a5f8 fffffa80`05d059e0 : Wdf01000+0x7859f
fffff880`023d3c50 fffff800`017d2827 : fffff880`023d3ce8 fffffa80`06206e20 fffffa80`06206e20 fffffa80`05d059e0 : Wdf01000+0x798de
fffff880`023d3c80 fffff800`014e2861 : fffff800`01741e00 fffff800`01741e80 fffff800`0167a5f8 fffffa80`05d059e0 : nt!ObCreateObject+0x767
fffff880`023d3cb0 fffff800`0177aa86 : fffff880`021ca570 fffffa80`05d059e0 00000000`00000080 fffffa80`04e7b450 : nt!KdPollBreakIn+0x2b1
fffff880`023d3d40 fffff800`014b3b06 : fffff800`0164fe80 fffffa80`05d059e0 fffffa80`04edab60 fffff880`0123ba90 : nt!PsCreateSystemThread+0x6f2
fffff880`023d3d80 00000000`00000000 : fffff880`023d4000 fffff880`023ce000 fffff880`023d39f0 00000000`00000000 : nt!KeTestAlertThread+0x93a


STACK_COMMAND:  kb

FOLLOWUP_IP: 
vioser+25d0
fffff880`029765d0 ff5018          call    qword ptr [rax+18h]

SYMBOL_STACK_INDEX:  7

SYMBOL_NAME:  vioser+25d0

FOLLOWUP_NAME:  MachineOwner

IMAGE_NAME:  vioser.sys

BUCKET_ID:  WRONG_SYMBOLS

Followup: MachineOwner
---------
Comment 1 Qunfang Zhang 2010-07-15 05:29:58 UTC 
Created attachment 431969 [details]
BSOD screenshot
Comment 2 Qunfang Zhang 2010-07-15 05:30:32 UTC 
Memory dump file:
http://10.66.65.120/mem-dump/MEMORY-2k8-R2-serial-1E.DMP
Comment 4 Yaniv Kaul 2010-07-15 05:42:16 UTC 
Try to get the right symbols:
BUCKET_ID:  WRONG_SYMBOLS
Comment 5 RHEL Product and Program Management 2010-07-15 14:17:09 UTC 
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 6 Vadim Rozenfeld 2010-07-15 17:16:49 UTC 
before we go further, can you try to install a fresh new OS with all default in-box drivers and try to run this test to detect whether we support Hot-Add CPU.

Two more things:
- was it a DataCenter?
- any chance to see DTM report?

Thanks,
Vadim.
Comment 7 Qunfang Zhang 2010-07-16 04:29:37 UTC 
(In reply to comment #6)
> before we go further, can you try to install a fresh new OS with all default
> in-box drivers and try to run this test to detect whether we support Hot-Add
> CPU.
Tried this, but there's no job "DPWLK-Hot-Add-Device Test-Verify driver support for Hot-Add CPU" if not load virtio-serial driver.

> 
> Two more things:
> - was it a DataCenter?
Yes, it's a Windows Server 2008 R2 Datacenter.

> - any chance to see DTM report?
The guest got BSOD and I reboot it manually, so there's no a DTM report. :(

> 
> Thanks,
> Vadim.
Comment 12 Qunfang Zhang 2010-07-16 09:18:37 UTC 
Microsoft (R) Windows Debugger Version 6.10.0003.233 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Administrator\Desktop\MEMORY-2k8-R2-serial.DMP]
Kernel Summary Dump File: Only kernel address space is available

Symbol search path is: SRV*C:\symbols\*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7600 MP (2 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
Built by: 7600.16539.amd64fre.win7_gdr.100226-1909
Machine Name:
Kernel base = 0xfffff800`01465000 PsLoadedModuleList = 0xfffff800`016a2e50
Debug session time: Thu Jul 15 04:52:30.953 2010 (GMT-7)
System Uptime: 0 days 0:07:04.031
Loading Kernel Symbols
...............................................................
............................................................
Loading User Symbols

Loading unloaded module list
.....
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 1E, {0, 0, 0, 0}

*** ERROR: Module load completed but symbols could not be loaded for vioser.sys
Probably caused by : vioser.sys ( vioser+25d0 )

Followup: MachineOwner
---------

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

KMODE_EXCEPTION_NOT_HANDLED (1e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: 0000000000000000, The exception code that was not handled
Arg2: 0000000000000000, The address that the exception occurred at
Arg3: 0000000000000000, Parameter 0 of the exception
Arg4: 0000000000000000, Parameter 1 of the exception

Debugging Details:
------------------


EXCEPTION_CODE: (Win32) 0 (0) - The operation completed successfully.

FAULTING_IP: 
+0
00000000`00000000 ??              ???

EXCEPTION_PARAMETER1:  0000000000000000

EXCEPTION_PARAMETER2:  0000000000000000

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0x1E

PROCESS_NAME:  System

CURRENT_IRQL:  2

EXCEPTION_RECORD:  fffff8800306eb68 -- (.exr 0xfffff8800306eb68)
ExceptionAddress: fffff880029765d0 (vioser+0x00000000000025d0)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: ffffffffffffffff
Attempt to read from address ffffffffffffffff

TRAP_FRAME:  fffff8800306ec10 -- (.trap 0xfffff8800306ec10)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa800638a000 rbx=0000000000000058 rcx=fffffa8005b0f000
rdx=fffff8800306edd0 rsi=0000000000000000 rdi=0000057ffabe8fd8
rip=fffff880029765d0 rsp=fffff8800306eda0 rbp=0000057ffabe8fd8
 r8=fffff88002979230  r9=0000000000000000 r10=fffffa80053f2e70
r11=fffff8800306edb0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei ng nz na po nc
vioser+0x25d0:
fffff880`029765d0 ff5018          call    qword ptr [rax+18h] ds:0001:fffffa80`0638a018=0002000000000000
Resetting default scope

LAST_CONTROL_TRANSFER:  from fffff800014cd32e to fffff800014d55d0

STACK_TEXT:  
fffff880`0306dc58 fffff800`014cd32e : 00000000`00000000 fffffa80`06089b60 fffff880`0306e3d0 fffff800`01502524 : nt!KeBugCheck
fffff880`0306dc60 fffff800`014fb2ed : fffff800`016e3b88 fffff800`0161ca40 fffff800`01465000 fffff880`0306eb68 : nt!KiKernelCalloutExceptionHandler+0xe
fffff880`0306dc90 fffff800`01502950 : fffff800`01624a50 fffff880`0306dd08 fffff880`0306eb68 fffff800`01465000 : nt!RtlpExecuteHandlerForException+0xd
fffff880`0306dcc0 fffff800`0150f8df : fffff880`0306eb68 fffff880`0306e3d0 fffff880`00000000 fffffa80`05aa8348 : nt!RtlDispatchException+0x410
fffff880`0306e3a0 fffff800`014d4c42 : fffff880`0306eb68 fffffa80`05b0f000 fffff880`0306ec10 fffffa80`05aa8348 : nt!KiDispatchException+0x16f
fffff880`0306ea30 fffff800`014d354a : fffff880`011798d0 00000000`00000058 fffff880`0306ed00 fffff880`0112e2dc : nt!KiExceptionDispatch+0xc2
fffff880`0306ec10 fffff880`029765d0 : 00000000`00000000 fffffa80`05acb240 fffffa80`062a0630 fffffa80`05417390 : nt!KiGeneralProtectionFault+0x10a
fffff880`0306eda0 fffff880`02976a7a : fffffa80`05acb240 fffffa80`05af4000 0000057f`fabe8fd8 fffffa80`05aa8348 : vioser+0x25d0
fffff880`0306edd0 fffff880`02975889 : fffffa80`062a0630 00000000`00000000 fffffa80`05b25000 00000000`00000002 : vioser+0x2a7a
fffff880`0306ee20 fffff880`011668a7 : fffffa80`00000008 0000057f`fab77fd8 fffffa80`053f1d02 fffffa80`053f1d20 : vioser+0x1889
fffff880`0306eed0 fffff800`014e0cdc : fffff880`041ef180 fffffa80`053f1d20 fffffa80`053f1d38 00000000`0000ffff : Wdf01000!FxInterrupt::_InterruptDpcThunk+0x8f
fffff880`0306ef00 fffff800`014db765 : 00000000`00000000 fffffa80`05d059e0 00000000`00000000 fffff880`01166818 : nt!KiRetireDpcList+0x1bc
fffff880`0306efb0 fffff800`014db57c : 00000000`000000a8 fffff880`02975777 fffffa80`05488020 00000000`00000000 : nt!KyRetireDpcList+0x5
fffff880`023d2fb0 fffff800`01520b13 : fffff800`014d1436 fffff800`014d14a2 00000000`000000a8 00000000`00000001 : nt!KiDispatchInterruptContinue
fffff880`023d2fe0 fffff800`014d14a2 : 00000000`000000a8 00000000`00000001 fffffa80`06361d80 00000000`00000000 : nt!KiDpcInterruptBypass+0x13
fffff880`023d2ff0 fffff800`01430b24 : 00000000`000000a8 fffff880`023d3300 00000000`00000001 fffff880`023d33f0 : nt!KiInterruptDispatch+0x212
fffff880`023d3180 fffff800`01427155 : fffff880`023d33d0 fffff880`023d330a fffff880`023d33d0 00000000`00000001 : hal!HalpEnableSystemInterrupt+0x1ac
fffff880`023d31e0 fffff800`015934eb : 00000000`0000000b 00000000`00000001 fffff880`023d33d0 00000000`00000002 : hal!HalEnableInterrupt+0x79
fffff880`023d3230 fffff800`018d750b : 00000000`00000003 00000000`00000002 00000000`00000001 fffffa80`061c87a0 : nt!KeConnectInterrupt+0x6b
fffff880`023d3270 fffff800`0190c8b4 : fffff880`023d34e0 fffff880`023d34e0 fffffa80`053f2e70 fffffa80`05480000 : nt!IopConnectInterrupt+0x31b
fffff880`023d3360 fffff880`01166a82 : fffffa80`05e3f428 fffffa80`05488020 fffffa80`05488090 fffff880`023d35b0 : nt!IoConnectInterruptEx+0x224
fffff880`023d3440 fffff880`01162247 : fffffa80`05488020 00000000`00000000 fffffa80`053be3c0 fffffa80`05488180 : Wdf01000!FxInterrupt::Connect+0x13a
fffff880`023d3540 fffff880`0115fa90 : fffffa80`053be020 00000000`00000000 fffff880`011745e0 fffffa80`053be020 : Wdf01000!FxPkgPnp::NotifyResourceObjectsD0+0x47
fffff880`023d35a0 fffff880`0115f6eb : 00000000`00000000 00000000`00000000 fffff880`011745c0 00000000`00000000 : Wdf01000!FxPkgPnp::PowerD0StartingConnectInterrupt+0x14
fffff880`023d3600 fffff880`0115f38e : fffffa80`053be020 00000000`00000040 fffff880`01174680 00000000`00000040 : Wdf01000!FxPkgPnp::PowerEnterNewState+0x1db
fffff880`023d3730 fffff880`0115f06c : 00000000`00000000 00000000`00000040 fffff880`01171da0 fffffa80`053be020 : Wdf01000!FxPkgPnp::PowerProcessEventInner+0x13e
fffff880`023d37a0 fffff880`01168c34 : 00000000`00000000 00000000`00000000 fffffa80`053be020 fffff880`01171da0 : Wdf01000!FxPkgPnp::PowerProcessEvent+0x1d4
fffff880`023d3830 fffff880`01168b37 : 00000000`00000000 00000000`00000000 fffff880`01171da0 fffff800`01664880 : Wdf01000!FxPkgPnp::PowerPolStarting+0x50
fffff880`023d3860 fffff880`01168691 : fffffa80`053be020 00000000`00000001 fffff880`011729e0 00000000`00000001 : Wdf01000!FxPkgPnp::PowerPolicyEnterNewState+0x1db
fffff880`023d3990 fffff880`01168372 : 00000000`00000000 00000000`00000001 00000000`00000118 00000000`00000118 : Wdf01000!FxPkgPnp::PowerPolicyProcessEventInner+0x139
fffff880`023d3a00 fffff880`011654f8 : 00000000`00000000 00000000`00000118 00000000`00000118 00000000`00000118 : Wdf01000!FxPkgPnp::PowerPolicyProcessEvent+0x1e2
fffff880`023d3a90 fffff880`01164841 : 00000000`00000118 00000000`0000011d 00000000`0000011d 00000000`00000000 : Wdf01000!FxPkgPnp::PnpEventRestarting+0x10
fffff880`023d3ac0 fffff880`011644fe : fffffa80`053be020 fffff880`023d3c20 00000000`00000004 fffff880`011736b0 : Wdf01000!FxPkgPnp::PnpEnterNewState+0x1a5
fffff880`023d3b30 fffff880`011643a4 : fffffa80`053be128 00000000`00000000 fffff880`023d3c20 fffffa80`053be020 : Wdf01000!FxPkgPnp::PnpProcessEventInner+0x122
fffff880`023d3ba0 fffff880`0116b59f : fffffa80`053be128 fffffa80`05d059e0 fffffa80`053be020 fffffa80`053be128 : Wdf01000!FxPkgPnp::_PnpProcessEventInner+0x60
fffff880`023d3be0 fffff880`0116c8de : 00000000`00000000 fffffa80`053ef450 fffff800`0167a5f8 fffffa80`05d059e0 : Wdf01000!FxEventQueue::EventQueueWorker+0x83
fffff880`023d3c50 fffff800`017d2827 : fffff880`023d3ce8 fffffa80`06206e20 fffffa80`06206e20 fffffa80`05d059e0 : Wdf01000!FxWorkItemEventQueue::_WorkItemCallback+0x1e
fffff880`023d3c80 fffff800`014e2861 : fffff800`01741e00 fffff800`01741e80 fffff800`0167a5f8 fffffa80`05d059e0 : nt!IopProcessWorkItem+0x23
fffff880`023d3cb0 fffff800`0177aa86 : fffff880`021ca570 fffffa80`05d059e0 00000000`00000080 fffffa80`04e7b450 : nt!ExpWorkerThread+0x111
fffff880`023d3d40 fffff800`014b3b06 : fffff800`0164fe80 fffffa80`05d059e0 fffffa80`04edab60 fffff880`0123ba90 : nt!PspSystemThreadStartup+0x5a
fffff880`023d3d80 00000000`00000000 : fffff880`023d4000 fffff880`023ce000 fffff880`023d39f0 00000000`00000000 : nt!KxStartSystemThread+0x16


STACK_COMMAND:  kb

FOLLOWUP_IP: 
vioser+25d0
fffff880`029765d0 ff5018          call    qword ptr [rax+18h]

SYMBOL_STACK_INDEX:  7

SYMBOL_NAME:  vioser+25d0

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: vioser

IMAGE_NAME:  vioser.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  4c2e1e75

FAILURE_BUCKET_ID:  X64_0x1E_vioser+25d0

BUCKET_ID:  X64_0x1E_vioser+25d0

Followup: MachineOwner
---------
Comment 13 Vadim Rozenfeld 2010-07-16 10:13:44 UTC 
You must be missing vioserial pdb file again. It should be like this: 



Microsoft (R) Windows Debugger Version 6.11.0001.404 X86

Copyright (c) Microsoft Corporation. All rights reserved.





Loading Dump File [Z:\dumps\vioserial614722\MEMORY-2k8-R2-serial-1E.DMP]

Kernel Summary Dump File: Only kernel address space is available



Symbol search path is: srv*C:\DownstreamStore*http://msdl.microsoft.com/download/symbols

Executable search path is: 

Windows 7 Kernel Version 7600 MP (2 procs) Free x64

Product: Server, suite: TerminalServer DataCenter SingleUserTS

Built by: 7600.16539.amd64fre.win7_gdr.100226-1909

Machine Name:

Kernel base = 0xfffff800`01465000 PsLoadedModuleList = 0xfffff800`016a2e50

Debug session time: Thu Jul 15 14:52:30.953 2010 (GMT+3)

System Uptime: 0 days 0:07:04.031

Loading Kernel Symbols

...............................................................

............................................................

Loading User Symbols



Loading unloaded module list

.....

*******************************************************************************

*                                                                             *

*                        Bugcheck Analysis                                    *

*                                                                             *

*******************************************************************************



Use !analyze -v to get detailed debugging information.



BugCheck 1E, {0, 0, 0, 0}



*** ERROR: Module load completed but symbols could not be loaded for vioser.sys

Probably caused by : vioser.sys ( vioser+25d0 )



Followup: MachineOwner

---------



1: kd> .reload

Loading Kernel Symbols

...............................................................

............................................................

Loading User Symbols



Loading unloaded module list

.....

1: kd> !analyze -v

*******************************************************************************

*                                                                             *

*                        Bugcheck Analysis                                    *

*                                                                             *

*******************************************************************************



KMODE_EXCEPTION_NOT_HANDLED (1e)

This is a very common bugcheck.  Usually the exception address pinpoints

the driver/function that caused the problem.  Always note this address

as well as the link date of the driver/image that contains this address.

Arguments:

Arg1: 0000000000000000, The exception code that was not handled

Arg2: 0000000000000000, The address that the exception occurred at

Arg3: 0000000000000000, Parameter 0 of the exception

Arg4: 0000000000000000, Parameter 1 of the exception



Debugging Details:

------------------





EXCEPTION_CODE: (Win32) 0 (0) - The operation completed successfully.



FAULTING_IP: 

+39b952f01c3dfdc

00000000`00000000 ??              ???



EXCEPTION_PARAMETER1:  0000000000000000



EXCEPTION_PARAMETER2:  0000000000000000



DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT



BUGCHECK_STR:  0x1E



PROCESS_NAME:  System



CURRENT_IRQL:  2



EXCEPTION_RECORD:  fffff8800306eb68 -- (.exr 0xfffff8800306eb68)

ExceptionAddress: fffff880029765d0 (vioser!VIOSerialReclaimConsumedBuffers+0x000000000000003c)

   ExceptionCode: c0000005 (Access violation)

  ExceptionFlags: 00000000

NumberParameters: 2

   Parameter[0]: 0000000000000000

   Parameter[1]: ffffffffffffffff

Attempt to read from address ffffffffffffffff



TRAP_FRAME:  fffff8800306ec10 -- (.trap 0xfffff8800306ec10)

NOTE: The trap frame does not contain all registers.

Some register values may be zeroed or incorrect.

rax=fffffa800638a000 rbx=0000000000000000 rcx=fffffa8005b0f000

rdx=fffff8800306edd0 rsi=0000000000000000 rdi=0000000000000000

rip=fffff880029765d0 rsp=fffff8800306eda0 rbp=0000057ffabe8fd8

 r8=fffff88002979230  r9=0000000000000000 r10=fffffa80053f2e70

r11=fffff8800306edb0 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei ng nz na po nc

vioser!VIOSerialReclaimConsumedBuffers+0x3c:

fffff880`029765d0 ff5018          call    qword ptr [rax+18h] ds:0001:fffffa80`0638a018=0002000000000000

Resetting default scope



LAST_CONTROL_TRANSFER:  from fffff800014cd32e to fffff800014d55d0



STACK_TEXT:  

fffff880`0306dc58 fffff800`014cd32e : 00000000`00000000 fffffa80`06089b60 fffff880`0306e3d0 fffff800`01502524 : nt!KeBugCheck

fffff880`0306dc60 fffff800`014fb2ed : fffff800`016e3b88 fffff800`0161ca40 fffff800`01465000 fffff880`0306eb68 : nt!KiKernelCalloutExceptionHandler+0xe

fffff880`0306dc90 fffff800`01502950 : fffff800`01624a50 fffff880`0306dd08 fffff880`0306eb68 fffff800`01465000 : nt!RtlpExecuteHandlerForException+0xd

fffff880`0306dcc0 fffff800`0150f8df : fffff880`0306eb68 fffff880`0306e3d0 fffff880`00000000 fffffa80`05aa8348 : nt!RtlDispatchException+0x410

fffff880`0306e3a0 fffff800`014d4c42 : fffff880`0306eb68 fffffa80`05b0f000 fffff880`0306ec10 fffffa80`05aa8348 : nt!KiDispatchException+0x16f

fffff880`0306ea30 fffff800`014d354a : fffff880`011798d0 00000000`00000058 fffff880`0306ed00 fffff880`0112e2dc : nt!KiExceptionDispatch+0xc2

fffff880`0306ec10 fffff880`029765d0 : 00000000`00000000 fffffa80`05acb240 fffffa80`062a0630 fffffa80`05417390 : nt!KiGeneralProtectionFault+0x10a

fffff880`0306eda0 fffff880`02976a7a : fffffa80`05acb240 fffffa80`05af4000 0000057f`fabe8fd8 fffffa80`05aa8348 : vioser!VIOSerialReclaimConsumedBuffers+0x3c [c:\cygwin\tmp\virtio-win-hdzd47gd\src\virtio-win-1.1.7-2\vioserial\sys\buffer.c @ 71]

fffff880`0306edd0 fffff880`02975889 : fffffa80`062a0630 00000000`00000000 fffffa80`05b25000 00000000`00000002 : vioser!VIOSerialHandleCtrlMsg+0x1ca [c:\cygwin\tmp\virtio-win-hdzd47gd\src\virtio-win-1.1.7-2\vioserial\sys\control.c @ 154]

fffff880`0306ee20 fffff880`011668a7 : fffffa80`00000008 0000057f`fab77fd8 fffffa80`053f1d02 fffffa80`053f1d20 : vioser!VIOSerialInterruptDpc+0xed [c:\cygwin\tmp\virtio-win-hdzd47gd\src\virtio-win-1.1.7-2\vioserial\sys\isrdpc.c @ 60]

fffff880`0306eed0 fffff800`014e0cdc : fffff880`041ef180 fffffa80`053f1d20 fffffa80`053f1d38 00000000`0000ffff : Wdf01000!FxInterrupt::_InterruptDpcThunk+0x8f

fffff880`0306ef00 fffff800`014db765 : 00000000`00000000 fffffa80`05d059e0 00000000`00000000 fffff880`01166818 : nt!KiRetireDpcList+0x1bc

fffff880`0306efb0 fffff800`014db57c : 00000000`000000a8 fffff880`02975777 fffffa80`05488020 00000000`00000000 : nt!KyRetireDpcList+0x5

fffff880`023d2fb0 fffff800`01520b13 : fffff800`014d1436 fffff800`014d14a2 00000000`000000a8 00000000`00000001 : nt!KiDispatchInterruptContinue

fffff880`023d2fe0 fffff800`014d14a2 : 00000000`000000a8 00000000`00000001 fffffa80`06361d80 00000000`00000000 : nt!KiDpcInterruptBypass+0x13

fffff880`023d2ff0 fffff800`01430b24 : 00000000`000000a8 fffff880`023d3300 00000000`00000001 fffff880`023d33f0 : nt!KiInterruptDispatch+0x212

fffff880`023d3180 fffff800`01427155 : fffff880`023d33d0 fffff880`023d330a fffff880`023d33d0 00000000`00000001 : hal!HalpEnableSystemInterrupt+0x1ac

fffff880`023d31e0 fffff800`015934eb : 00000000`0000000b 00000000`00000001 fffff880`023d33d0 00000000`00000002 : hal!HalEnableInterrupt+0x79

fffff880`023d3230 fffff800`018d750b : 00000000`00000003 00000000`00000002 00000000`00000001 fffffa80`061c87a0 : nt!KeConnectInterrupt+0x6b

fffff880`023d3270 fffff800`0190c8b4 : fffff880`023d34e0 fffff880`023d34e0 fffffa80`053f2e70 fffffa80`05480000 : nt!IopConnectInterrupt+0x31b

fffff880`023d3360 fffff880`01166a82 : fffffa80`05e3f428 fffffa80`05488020 fffffa80`05488090 fffff880`023d35b0 : nt!IoConnectInterruptEx+0x224

fffff880`023d3440 fffff880`01162247 : fffffa80`05488020 00000000`00000000 fffffa80`053be3c0 fffffa80`05488180 : Wdf01000!FxInterrupt::Connect+0x13a

fffff880`023d3540 fffff880`0115fa90 : fffffa80`053be020 00000000`00000000 fffff880`011745e0 fffffa80`053be020 : Wdf01000!FxPkgPnp::NotifyResourceObjectsD0+0x47

fffff880`023d35a0 fffff880`0115f6eb : 00000000`00000000 00000000`00000000 fffff880`011745c0 00000000`00000000 : Wdf01000!FxPkgPnp::PowerD0StartingConnectInterrupt+0x14

fffff880`023d3600 fffff880`0115f38e : fffffa80`053be020 00000000`00000040 fffff880`01174680 00000000`00000040 : Wdf01000!FxPkgPnp::PowerEnterNewState+0x1db

fffff880`023d3730 fffff880`0115f06c : 00000000`00000000 00000000`00000040 fffff880`01171da0 fffffa80`053be020 : Wdf01000!FxPkgPnp::PowerProcessEventInner+0x13e

fffff880`023d37a0 fffff880`01168c34 : 00000000`00000000 00000000`00000000 fffffa80`053be020 fffff880`01171da0 : Wdf01000!FxPkgPnp::PowerProcessEvent+0x1d4

fffff880`023d3830 fffff880`01168b37 : 00000000`00000000 00000000`00000000 fffff880`01171da0 fffff800`01664880 : Wdf01000!FxPkgPnp::PowerPolStarting+0x50

fffff880`023d3860 fffff880`01168691 : fffffa80`053be020 00000000`00000001 fffff880`011729e0 00000000`00000001 : Wdf01000!FxPkgPnp::PowerPolicyEnterNewState+0x1db

fffff880`023d3990 fffff880`01168372 : 00000000`00000000 00000000`00000001 00000000`00000118 00000000`00000118 : Wdf01000!FxPkgPnp::PowerPolicyProcessEventInner+0x139

fffff880`023d3a00 fffff880`011654f8 : 00000000`00000000 00000000`00000118 00000000`00000118 00000000`00000118 : Wdf01000!FxPkgPnp::PowerPolicyProcessEvent+0x1e2

fffff880`023d3a90 fffff880`01164841 : 00000000`00000118 00000000`0000011d 00000000`0000011d 00000000`00000000 : Wdf01000!FxPkgPnp::PnpEventRestarting+0x10

fffff880`023d3ac0 fffff880`011644fe : fffffa80`053be020 fffff880`023d3c20 00000000`00000004 fffff880`011736b0 : Wdf01000!FxPkgPnp::PnpEnterNewState+0x1a5

fffff880`023d3b30 fffff880`011643a4 : fffffa80`053be128 00000000`00000000 fffff880`023d3c20 fffffa80`053be020 : Wdf01000!FxPkgPnp::PnpProcessEventInner+0x122

fffff880`023d3ba0 fffff880`0116b59f : fffffa80`053be128 fffffa80`05d059e0 fffffa80`053be020 fffffa80`053be128 : Wdf01000!FxPkgPnp::_PnpProcessEventInner+0x60

fffff880`023d3be0 fffff880`0116c8de : 00000000`00000000 fffffa80`053ef450 fffff800`0167a5f8 fffffa80`05d059e0 : Wdf01000!FxEventQueue::EventQueueWorker+0x83

fffff880`023d3c50 fffff800`017d2827 : fffff880`023d3ce8 fffffa80`06206e20 fffffa80`06206e20 fffffa80`05d059e0 : Wdf01000!FxWorkItemEventQueue::_WorkItemCallback+0x1e

fffff880`023d3c80 fffff800`014e2861 : fffff800`01741e00 fffff800`01741e80 fffff800`0167a5f8 fffffa80`05d059e0 : nt!IopProcessWorkItem+0x23

fffff880`023d3cb0 fffff800`0177aa86 : fffff880`021ca570 fffffa80`05d059e0 00000000`00000080 fffffa80`04e7b450 : nt!ExpWorkerThread+0x111

fffff880`023d3d40 fffff800`014b3b06 : fffff800`0164fe80 fffffa80`05d059e0 fffffa80`04edab60 fffff880`0123ba90 : nt!PspSystemThreadStartup+0x5a

fffff880`023d3d80 00000000`00000000 : fffff880`023d4000 fffff880`023ce000 fffff880`023d39f0 00000000`00000000 : nt!KxStartSystemThread+0x16





STACK_COMMAND:  kb



FOLLOWUP_IP: 

vioser!VIOSerialReclaimConsumedBuffers+3c [c:\cygwin\tmp\virtio-win-hdzd47gd\src\virtio-win-1.1.7-2\vioserial\sys\buffer.c @ 71]

fffff880`029765d0 ff5018          call    qword ptr [rax+18h]



FAULTING_SOURCE_CODE:  

    67:     while ((buf = vq->vq_ops->get_buf(vq, &len)) != NULL)

    68:     {

    69:         ExFreePoolWithTag(buf, VIOSERIAL_DRIVER_MEMORY_TAG);  

    70:         KeStallExecutionProcessor(100);

>   71:         port->OutVqFull = FALSE;

    72:     }

    73:     TraceEvents(TRACE_LEVEL_INFORMATION, DBG_PNP, "<-- %s port->OutVqFull = %d\n", __FUNCTION__, port->OutVqFull);

    74: }

    75: 

    76: SSIZE_T 





SYMBOL_STACK_INDEX:  7



SYMBOL_NAME:  vioser!VIOSerialReclaimConsumedBuffers+3c



FOLLOWUP_NAME:  MachineOwner



MODULE_NAME: vioser



IMAGE_NAME:  vioser.sys



DEBUG_FLR_IMAGE_TIMESTAMP:  4c2e1e75



FAILURE_BUCKET_ID:  X64_0x1E_vioser!VIOSerialReclaimConsumedBuffers+3c



BUCKET_ID:  X64_0x1E_vioser!VIOSerialReclaimConsumedBuffers+3c



Followup: MachineOwner

---------



1: kd> .trap 0xfffff8800306ec10

NOTE: The trap frame does not contain all registers.

Some register values may be zeroed or incorrect.

rax=fffffa800638a000 rbx=0000000000000000 rcx=fffffa8005b0f000

rdx=fffff8800306edd0 rsi=0000000000000000 rdi=0000000000000000

rip=fffff880029765d0 rsp=fffff8800306eda0 rbp=0000057ffabe8fd8

 r8=fffff88002979230  r9=0000000000000000 r10=fffffa80053f2e70

r11=fffff8800306edb0 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei ng nz na po nc

vioser!VIOSerialReclaimConsumedBuffers+0x3c:

fffff880`029765d0 ff5018          call    qword ptr [rax+18h] ds:0001:fffffa80`0638a018=0002000000000000
The dump is absolutely identical to another one, reported in #613949.
So, we have two different scenarios to reproduce this bug.
Comment 14 Vadim Rozenfeld 2010-08-03 07:30:11 UTC 
Hi, Qunfang.

Can you please re-check this issue with the lates driver?

Thanks,
Vadim.
Comment 15 Qunfang Zhang 2010-08-03 09:55:18 UTC 
(In reply to comment #14)
> Hi, Qunfang.
> 
> Can you please re-check this issue with the lates driver?
> 
> Thanks,
> Vadim.    

Hi, Vadim

Re-test with the latest driver 1.1.10-0, this issue still exists with the same error code 0x1E.

virtio-win-1.1.10-0
qemu-kvm-0.12.1.2-2.104.el6.x86_64
2.6.32-54.el6.x86_64
(use the 20100730.5 install tree in host.)
Comment 16 Qunfang Zhang 2010-08-03 09:56:18 UTC 
I will change the status back according to Comment 15, thanks.
Comment 17 Vadim Rozenfeld 2010-08-03 12:47:39 UTC 
(In reply to comment #16)
> I will change the status back according to Comment 15, thanks.    

Thanks, Qunfang.
Please post the crash dump file.
Regards,
Vadim.
Comment 18 Qunfang Zhang 2010-08-04 05:18:03 UTC 
(In reply to comment #17)
> (In reply to comment #16)
> > I will change the status back according to Comment 15, thanks.    
> 
> Thanks, Qunfang.
> Please post the crash dump file.
> Regards,
> Vadim.    

Hi, Vadim

dump file:
http://10.66.65.120/mem-dump/MEMORY-win2k8-R2-serial-0x1E-bz614722.DMP.gz
dump analyze file:
http://10.66.65.120/mem-dump/MEMORY-win2k8-R2-serial-0x1E-bz614722.txt

Regards~
Qunfang
Comment 20 Qunfang Zhang 2010-08-23 08:25:36 UTC 
Update:
The issue does not exist on virtio-win-1.1.12-0 using win2k8-R2.
I will change the status to VERIFIED after finish all windows guests.
Comment 21 Qunfang Zhang 2010-08-26 12:17:49 UTC 
Verified this bug on virio-win-1.1.12-0 with win2k8-R2 and win2k8-64 guests.
This issue does not exist any more.
For other guest os, have no this job.
Related package version:
kernel-2.6.32-66.el6.x86_64
qemu-kvm-0.12.1.2-2.112.el6.x86_64

So, I will change the status to VERIFIED.
Comment 22 releng-rhel@redhat.com 2010-11-11 15:01:46 UTC 
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.
Note You need to log in before you can comment on or make changes to this bug.