Bug 858097 - [whql][netkvm]MPE job failed on windows 7 guest with BSOD 7E error code
[whql][netkvm]MPE job failed on windows 7 guest with BSOD 7E error code
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: virtio-win (Show other bugs)
6.5
Unspecified Unspecified
high Severity high
: rc
: ---
Assigned To: Yan Vugenfirer
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-17 20:31 EDT by Min Deng
Modified: 2013-12-06 02:14 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-01-08 07:46:25 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
DUMP-analysis (13.26 KB, application/vnd.oasis.opendocument.text)
2012-09-17 20:44 EDT, Min Deng
no flags Details
Analysis (32.30 KB, application/octet-stream)
2012-10-09 04:30 EDT, Min Deng
no flags Details

  None (edit)
Description Min Deng 2012-09-17 20:31:53 EDT
Description of problem:
One BSOD 7E error occurred on windows 7 32/64 bits,both VNC and SPICE can help to reproduce the issue.

Version-Release number of selected component (if applicable):
kernel-2.6.32-302.el6.x86_64
qemu-kvm-0.12.1.2-2.307.el6.x86_64
virti-win-prewhql-0.1-35

How reproducible:
5 time failed 5 times 
Steps to Reproduce:
1.boot up guest with CLI
  guest1:/usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -drive file=win7-32-nic2.raw,format=raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,sndbuf=0,id=hostnet0,vhost=on,script=/etc/qemu-ifup-private,downscript=no -device virtio-net-pci,netdev=hostnet0,mac=00:22:37:01:19:ca,bus=pci.0,addr=0x4,id=virtio-net-pci0 -netdev tap,sndbuf=0,id=hostnet2,script=/etc/qemu-ifup,downscript=no -device e1000,netdev=hostnet2,mac=00:22:40:02:1c:43,bus=pci.0,addr=0x6 -uuid 91c67057-42db-4ab1-a4a1-7adddf73ac61 -no-kvm-pit-reinjection -chardev socket,id=111b,path=/tmp/monitor-win7-32-nic2,server,nowait -mon chardev=111b,mode=readline -spice port=5932,disable-ticketing -vga qxl -rtc base=localtime,clock=host,driftfix=slew -bios /usr/share/seabios/bios-pm.bin

2.guest2:/usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -drive file=win7-32-nic1.raw,format=raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,sndbuf=0,id=hostnet0,vhost=on,script=/etc/qemu-ifup-private,downscript=no -device virtio-net-pci,netdev=hostnet0,mac=00:22:15:3a:40:43,bus=pci.0,addr=0x4,id=virtio-net-pci0 -netdev tap,sndbuf=0,id=hostnet1,vhost=on,script=/etc/qemu-ifup-private,downscript=no -device virtio-net-pci,netdev=hostnet1,mac=00:22:21:46:22:a1,bus=pci.0,addr=0x5,id=virtio-net-pci1 -netdev tap,sndbuf=0,id=hostnet2,script=/etc/qemu-ifup,downscript=no -device e1000,netdev=hostnet2,mac=00:22:13:15:14:22,bus=pci.0,addr=0x6 -uuid 9e6b89dc-7082-4e5d-91be-0da461770f90 -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-win7-32-nic1,server,nowait -mon chardev=111a,mode=readline -spice port=5931,disable-ticketing -vga qxl -rtc base=localtime,clock=host,driftfix=slew -bios /usr/share/seabios/bios-pm.bin
2.submit MPE job on HCK

Actual results:
The job failed with BSOD error code 7E
Expected results:
The job passed without error
Additional info:
will upload dump files later
Comment 2 Min Deng 2012-09-17 20:44:14 EDT
Created attachment 613831 [details]
DUMP-analysis
Comment 5 Mike Cao 2012-09-20 05:55:59 EDT
Yan, According to the backtrace ,the BSOD caused by ndprot62.sys , Need we open a case to MSFT ?

Best Regards,
Mike
Comment 6 Yan Vugenfirer 2012-10-02 06:08:51 EDT
Please retest with build 39.
Comment 7 Min Deng 2012-10-09 04:30:06 EDT
Fortunately,I still can reproduce the issue.Both win732 and win764 hit the bug via build 39 and please see the latest analysis of dump.
Comment 8 Min Deng 2012-10-09 04:30:41 EDT
Created attachment 623974 [details]
Analysis
Comment 9 Yan Vugenfirer 2012-10-09 05:39:20 EDT
Can you please upload zipped dump file?

Thanks!
Comment 10 Yan Vugenfirer 2012-10-09 08:59:16 EDT
Hi Mike,

I suggest to forward it to Microsoft.

Here are several things I see.

First of all the NICs are in paused state (both Intel e1000 and Virtio). 
You can check it by using following command in windbag:
!ndiskd.miniports - it will show the miniports and their handles
!ndiskd.miniport <specific miniport handle or double click on the handle you see as the output of !ndiskd.miniports>

Also - checking the stack:

ndprot62 tried to send some data (but how can it do it, NICs are paused):
ndprot62!CNDTSimpleTrafficMgr::AcquireSendToken

and then it debug breaks and here is the address of the string it tries to print out:


99bab668 4e 44 49 53 54 65 73 74 3a 20 28 42 29 72 65 61 6b 2c 20  NDISTest: (B)reak, 
99bab67b 42 72 65 61 6b 20 28 41 29 6c 77 61 79 73 2c 20 28 49 29  Break (A)lways, (I)
99bab68e 67 6e 6f 72 65 2c 20 49 67 6e 6f 72 65 20 41 28 6c 29 77  gnore, Ignore A(l)w
99bab6a1 61 79 73 2c 20 28 47 29 6f 3a 20 00 cc 54 6f 6f 20 6d 61  ays, (G)o: ..Too ma
99bab6b4 6e 79 20 75 6e 61 63 63 65 70 74 61 62 6c 65 20 63 68 61  ny unacceptable cha
99bab6c7 72 61 63 74 65 72 73 2e 20 42 72 65 61 6b 69 6e 67 20 61  racters. Breaking a
99bab6da 6e 79 77 61 79 73 0a 00 74 65 73 74 73 72 63 5c 6e 65 74  nyways..testsrc\net
99bab6ed 74 65 73 74 5c 6e 64 69 73 5c 6e 64 69 73 74 65 73 74 5c  test\ndis\ndistest\
99bab700 64 72 76 73 68 65 6c 6c 5c 73 72 63 5c 63 6f 6d 6d 61 6e  drvshell\src\comman
99bab713 64 70 72 6f 63 65 73 73 6f 72 2e 63 70 70 00 70 54 61 72  dprocessor.cpp.pTar
99bab726 67 65 74 43 61 6c 6c 42 61 63 6b 20 21 3d 20 30 00 cc 21  getCallBack != 0..!
99bab739 6d 5f 54 61 72 67 65 74 54 61 62 6c 65 5b 75 6c 54 61 72  m_TargetTable[ulTar
99bab74c 67 65 74 43 6f 64 65 5d 00 cc 43 4e 44 54 43 6f 6d 6d 61  getCode]..CNDTComma
99bab75f 6e 64 50 72 6f 63 65 73 73 6f 72 3a 3a 50 72 6f 63 65 73  ndProcessor::Proces
99bab772 73 43 6f 6d 6d 61 6e 64 20 77 61 73 20 49 6e 76 6f 6b 65  sCommand was Invoke
99bab785 64 20 61 74 20 49 52 51 4c 20 3e 20 50 41 53 53 49 56 45  d at IRQL > PASSIVE
99bab798 21 20 54 68 69 73 20 69 73 20 61 20 46 61 74 61 6c 20 45  ! This is a Fatal E
99bab7ab 72 72 6f 72 0a 00 cc 43 4e 44 54 43 6f 6d 6d 61 6e 64 50  rror...
Comment 11 Yan Vugenfirer 2012-10-09 09:04:53 EDT
According to the MEMORY-764 dump (also here NICs are in paused states):

fffff880`04f6ce00 74 65 73 74 73 72 63 5c 6e 65 74 74 65 73 74 5c 6e 64 69 73 5c 6e 64 69 73 74 65 73 74 5c  testsrc\nettest\ndis\ndistest\
fffff880`04f6ce1e 63 6f 6d 6d 65 6e 67 69 6e 65 5c 6f 74 68 65 72 73 5c 73 69 6d 70 6c 65 74 72 61 66 66 69  commengine\others\simpletraffi
fffff880`04f6ce3c 63 6d 61 6e 61 67 65 72 2e 63 70 70 00 cc cc cc cc cc cc cc 43 4e 44 54 53 69 6d 70 6c 65  cmanager.cpp........CNDTSimple
fffff880`04f6ce5a 54 72 61 66 66 69 63 4d 67 72 3a 3a 4f 70 65 6e 43 6f 6d 6d 75 6e 69 63 61 74 69 6f 6e 43  TrafficMgr::OpenCommunicationC
fffff880`04f6ce78 68 61 6e 6e 65 6c 3a 20 46 61 69 6c 65 64 20 74 6f 20 61 6c 6c 6f 63 61 74 65 20 4e 65 74  hannel: Failed to allocate Net
fffff880`04f6ce96 42 75 66 66 65 72 4d 6f 64 75 6c 65 53 74 61 63 6b 2e 20 53 79 73 74 65 6d 20 6d 61 79 20  BufferModuleStack. System may 
fffff880`04f6ceb4 62 65 20 72 75 6e 6e 69 6e 67 20 6c 6f 77 20 6f 6e 20 72 65 73 6f 75 72 63 65 73 0a 00 cc  be running low on resources.
Comment 13 Mike Cao 2013-01-07 04:55:00 EST
10/30/2012 15:55
Subject:Re: [REG:112101024393405] BSOD occurs when running MPE jobs for win7-32/64  Initial Response
 
Hi, Rohit

After We use fresh installed image to retest it ,the job can pass successfully
Thanks for your hard working on this case ,feel free to close it

Best Regards,
Mike

Quality Assurance Engineer
Desk No: +86 010 6260 8241
Mobile No: +86 18610633772
Time Zone: UTC +8 beijing
Red Hat Software (Beijing) Co.
9/F, North Tower C, Raycom Infotech Park
No.2 Ke Xueyuan Nanlu, Haidian District Beijing 100190

Note You need to log in before you can comment on or make changes to this bug.