Bug 1234751

Summary: [virtio-win][vioscsi]win2012R2 guest bsod(d1) when whql test DPWLK-HotAdd(1104) job
Product: Red Hat Enterprise Linux 7 Reporter: Yu Wang <wyu>
Component: virtio-winAssignee: Vadim Rozenfeld <vrozenfe>
virtio-win sub component: virtio-win-prewhql QA Contact: Virtualization Bugs <virt-bugs>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: unspecified CC: lijin, lmiksik, wyu
Version: 7.2Keywords: TestBlocker
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
NO_DOCS
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 08:46:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1288337    

Description Yu Wang 2015-06-23 07:57:33 UTC
Description of problem:
win2012R2 guest bsod(d1) when whql test DPWLK-HotAdd(1104) job

Version-Release number of selected component (if applicable):

kernel-debug-3.10.0-267.el7.x86_64
qemu-kvm-rhev-2.3.0-2.el7.x86_64
seabios-bin-1.7.5-9.el7.x86_64
virtio-win-prewhql-105

How reproducible:
100%

Steps to Reproduce:
1.boot guest with:
/usr/libexec/qemu-kvm -name 105SCS2012R2Z9N -enable-kvm -m 3G -smp 4 -uuid 725933ed-17d6-4f69-9242-956feb449b22 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/tmp/105SCS2012R2Z9N,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=105SCS2012R2Z9N,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_server_2012_r2_x64_dvd_2707946.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=105SCS2012R2Z9N.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none -global isa-fdc.driveA=drive-fdc0-0-0 -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:52:44:14:08:9a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:1 -vga cirrus -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7,num_queues=8 -drive file=105SCS2012R2Z9N_test.raw,if=none,id=drive-scsi-disk0,format=raw,serial=mike_cao,cache=none -device scsi-hd,bus=scsi0.0,drive=drive-scsi-disk0,id=scsi-disk0

2.run whql test DPWLK-HotAdd(1104) job

Actual results:
bsod(d1)

Expected results:
Pass

Additional info:

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: ffffd00020be7000, memory referenced
Arg2: 000000000000000b, IRQL
Arg3: 0000000000000001, value 0 = read operation, 1 = write operation
Arg4: fffff80000840060, address which referenced memory

Debugging Details:
------------------


WRITE_ADDRESS:  ffffd00020be7000 

CURRENT_IRQL:  b

FAULTING_IP: 
vioscsi+1060
fffff800`00840060 488911          mov     qword ptr [rcx],rdx

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  AV

PROCESS_NAME:  System

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre

TRAP_FRAME:  ffffd00020946140 -- (.trap 0xffffd00020946140)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=ffffd00020be7000 rbx=0000000000000000 rcx=ffffd00020be7000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffff80000840060 rsp=ffffd000209462d8 rbp=ffffd00020946390
 r8=0000000000000000  r9=0000000000000005 r10=ffffd00020be3000
r11=ffffd00020946260 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
vioscsi+0x1060:
fffff800`00840060 488911          mov     qword ptr [rcx],rdx ds:ffffd000`20be7000=????????????????
Resetting default scope

LOCK_ADDRESS:  fffff800da7580e0 -- (!locks fffff800da7580e0)

Resource @ nt!PiEngineLock (0xfffff800da7580e0)    Exclusively owned
    Contention Count = 6
    NumberOfExclusiveWaiters = 1
     Threads: ffffe000011f2280-01<*> 
     Threads Waiting On Exclusive Access:
              ffffe000011f3400       

1 total locks, 1 locks currently held

PNP_TRIAGE: 
	Lock address  : 0xfffff800da7580e0
	Thread Count  : 1
	Thread address: 0xffffe000011f2280
	Thread wait   : 0x7658

LAST_CONTROL_TRANSFER:  from fffff800da5e1be9 to fffff800da5d60a0

STACK_TEXT:  
ffffd000`20945ff8 fffff800`da5e1be9 : 00000000`0000000a ffffd000`20be7000 00000000`0000000b 00000000`00000001 : nt!KeBugCheckEx
ffffd000`20946000 fffff800`da5e043a : 00000000`00000001 ffffd000`20be7000 00000000`00000000 ffffd000`20946140 : nt!KiBugCheckDispatch+0x69
ffffd000`20946140 fffff800`00840060 : fffff800`00840e86 00000000`00000000 00000000`00000008 00000000`00000003 : nt!KiPageFault+0x23a
ffffd000`209462d8 fffff800`00840e86 : 00000000`00000000 00000000`00000008 00000000`00000003 00000000`00000004 : vioscsi+0x1060
ffffd000`209462e0 fffff800`00842093 : ffffe000`00003000 ffffe000`01a2b020 00000000`0000000b ffffd000`00004000 : vioscsi+0x1e86
ffffd000`20946310 fffff800`0085a3f0 : a2bd4c89`0b6aa2a8 ffffe000`01a261a0 00000000`00000000 00000000`00000040 : vioscsi+0x3093
ffffd000`209463c0 fffff800`0085e373 : ffffe000`01a261a0 00000000`00000040 ffffcf80`00aa6d80 00000000`00000040 : storport!RaCallMiniportHwInitialize+0x1c
ffffd000`209463f0 fffff800`0085de8a : ffffe000`011fee90 00000000`00000040 ffffe000`01a261a0 ffffcf80`00aa6d80 : storport!RaidAdapterStartMiniport+0x113
ffffd000`20946450 fffff800`008568d6 : ffffe000`00000000 fffff800`da760ec0 ffffcf80`00aa6d80 ffffd000`20946540 : storport!RaidAdapterStartDeviceIrp+0x116
ffffd000`209464d0 fffff800`0084fde1 : ffffe000`027aa010 fffff800`0029cbfb ffffe000`01a26050 ffffe000`01a26050 : storport!RaidAdapterPnpIrp+0x1ae
ffffd000`20946570 fffff800`00297c12 : ffffe000`01a26050 ffffd000`209465e0 ffffcf80`00aa6d80 fffff800`0084fd54 : storport!RaDriverPnpIrp+0x8d
ffffd000`209465b0 fffff800`daae9911 : ffffcf80`00aa6d80 00000000`00000002 ffffe000`027d1c10 00000000`00000000 : VerifierExt!xdv_IRP_MJ_PNP_wrapper+0xfe
ffffd000`20946610 fffff800`dab060f8 : ffffcf80`00aa6d80 ffffe000`01a2bca0 ffffe000`01a2bca0 ffffe000`00c7d3d0 : nt!IovCallDriver+0x3cd
ffffd000`20946660 fffff800`daae9911 : ffffe000`01a2bdf0 ffffcf80`00aa6d80 ffffe000`01a2bca0 00000000`00000002 : nt!ViFilterDispatchPnp+0x22c
ffffd000`209466a0 fffff800`da89b072 : ffffcf80`00aa6d80 ffffd000`20946858 ffffe000`01a2bca0 ffffe000`00c7d230 : nt!IovCallDriver+0x3cd
ffffd000`209466f0 fffff800`da567539 : ffffe000`011d4060 ffffd000`20946799 00000000`00000000 fffff800`da728100 : nt!PnpAsynchronousCall+0x102
ffffd000`20946730 fffff800`daa00c9c : ffffe000`011d6490 ffffe000`011d6490 00000000`00000000 fffff800`da88aa46 : nt!PnpStartDevice+0xc5
ffffd000`20946800 fffff800`daa72ad1 : ffffd000`20946960 ffffe000`011d6490 00000000`00000001 00000000`00000000 : nt! ?? ::NNGAKEGL::`string'+0x4e09c
ffffd000`209468d0 fffff800`da9e44b0 : ffffe000`011d6490 00000000`00000001 00000000`00000002 ffffe000`0011fd30 : nt!PipProcessRestartPhase1+0x65
ffffd000`20946910 fffff800`da65a376 : 00000000`00000000 ffffe000`00000001 ffffe000`00000000 ffffe000`02bcb000 : nt! ?? ::NNGAKEGL::`string'+0x318b0
ffffd000`20946b90 fffff800`da614c22 : fffff801`00000003 00000000`00000000 00000000`00000000 00000000`00000000 : nt!PnpProcessRebalance+0x142
ffffd000`20946be0 fffff800`da4c23cd : fffff800`da53cd34 ffffd000`20946d10 00000000`00000000 ffffcf80`004acfb0 : nt! ?? ::FNODOBFM::`string'+0x2e692
ffffd000`20946c90 fffff800`da56d664 : 00000000`00010000 ffffe000`011f2280 ffffe000`011f2280 ffffe000`000c0780 : nt!ExpWorkerThread+0x2b5
ffffd000`20946d40 fffff800`da5dc6c6 : fffff800`da777180 ffffe000`011f2280 ffffe000`0016b040 00000020`52545348 : nt!PspSystemThreadStartup+0x58
ffffd000`20946da0 00000000`00000000 : ffffd000`20947000 ffffd000`20941000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16


STACK_COMMAND:  kb

FOLLOWUP_IP: 
vioscsi+1060
fffff800`00840060 488911          mov     qword ptr [rcx],rdx

SYMBOL_STACK_INDEX:  3

SYMBOL_NAME:  vioscsi+1060

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: vioscsi

IMAGE_NAME:  vioscsi.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  556eddd9

FAILURE_BUCKET_ID:  AV_VRF_vioscsi+1060

BUCKET_ID:  AV_VRF_vioscsi+1060

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:av_vrf_vioscsi+1060

FAILURE_ID_HASH:  {14cae5e0-0246-7a1c-061b-86f805d65624}

Followup: MachineOwner
---------

Comment 1 Yu Wang 2015-06-23 08:01:41 UTC
The memory.dump file is located at:
http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/virtio-win/bug1234751/

Comment 6 Vadim Rozenfeld 2015-07-30 10:52:03 UTC
Please re-check with drivers from build 107
http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/107/win/virtio-win-prewhql-0.1-107-spec.zip

Comment 7 lijin 2015-09-11 05:25:40 UTC
change status to verified as build 109 scsi whql jobs all pass

Comment 8 Vadim Rozenfeld 2015-09-30 06:02:28 UTC
This bug is tightly coupled with https://bugzilla.redhat.com/show_bug.cgi?id=1210166 
Moving it to 7.3 as well.

Comment 11 Yu Wang 2016-07-13 06:42:54 UTC
According to our vioscsi WHQL Test result for virtio-win-prewhql-121, this job has passed, so this bug has been fixed.

Change status to verified.

Thanks
Yu Wang

Comment 13 errata-xmlrpc 2016-11-04 08:46:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2609.html