Bug 873192
| Summary: | [whql][Netkvm] Job "NDISTest6.5-InvalidPackets" failed with WLK on win2k8/win2k8-R2 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | dawu | ||||||||
| Component: | virtio-win | Assignee: | Dmitry Fleytman <dfleytma> | ||||||||
| Status: | CLOSED WORKSFORME | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||||
| Severity: | high | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 6.4 | CC: | acathrow, bcao, bsarathy, dfleytma, dyasny, juzhang, lijin, michen, rhod, yvugenfi | ||||||||
| Target Milestone: | rc | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2012-12-17 08:32:39 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Created attachment 639059 [details]
cpk-win2k8-32-network-43.cpk
Please refer to the attached for the cpk file.
Best Regards,
Dawn
We cannot find the exact cause of failure from the logs... Do you see that the drivers are correctly installed on the VM? Best regards, Yan. In any case - please retest with: http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/44/win/ (this is also a fix for https://bugzilla.redhat.com/show_bug.cgi?id=873128 ) (In reply to comment #3) > We cannot find the exact cause of failure from the logs... > > Do you see that the drivers are correctly installed on the VM? Yes, the drivers are correctly installed on the VM, I retest again today with keeping eyes on the guests,find when the NDISTest Client is doing the last job of "postconfig", the NDISTest Server side finish "postconfig" job before Client and quickly quit.But the Client is still running "postconfig". And another thing is job "InvalidPackets.htm" fail for NDIS6.5, please refer to the attachment "InvalidPacketsFail.png" Any way, I'll retest it on new driver 44, and update the result. Thanks Best Regards, Dawn > > Best regards, > Yan. Created attachment 645562 [details]
InvalidPacketsFail
Is it possible to run same test with REHL 6.3 host? Thanks, Yan. (In reply to comment #7) > Is it possible to run same test with REHL 6.3 host? > > Thanks, > Yan. ongoing now (In reply to comment #6) > Created attachment 645562 [details] > InvalidPacketsFail Can you double click on the test in the UI list and it should open the test log. Please save it and attach to BZ. Thanks, Yan. (In reply to comment #9) > (In reply to comment #6) > > Created attachment 645562 [details] > > InvalidPacketsFail > > Can you double click on the test in the UI list and it should open the test > log. Please save it and attach to BZ. > > Thanks, > Yan. (In reply to comment #9) > (In reply to comment #6) > > Created attachment 645562 [details] > > InvalidPacketsFail > > Can you double click on the test in the UI list and it should open the test > log. Please save it and attach to BZ. > > Thanks, > Yan. Hi Yan, Please refer to the attachment of "InvalidPackets.htm" for details. Best Regards, Dawn Created attachment 653272 [details]
InvalidPackets.htm
(In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #6) > > > Created attachment 645562 [details] > > > InvalidPacketsFail > > > > Can you double click on the test in the UI list and it should open the test > > log. Please save it and attach to BZ. > > > > Thanks, > > Yan. > > (In reply to comment #9) > > (In reply to comment #6) > > > Created attachment 645562 [details] > > > InvalidPacketsFail > > > > Can you double click on the test in the UI list and it should open the test > > log. Please save it and attach to BZ. > > > > Thanks, > > Yan. > > Hi Yan, > > Please refer to the attachment of "InvalidPackets.htm" for details. > > Best Regards, > Dawn Thanks! Any news on using RHEL6.3 host? Bridge filtering settings: Good settings are: # sysctl -a|grep bridge net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 Hi Yan, Tried to run the same test with rhel6.3 host, still hit the same issue with rhel6.4 host. Best Regards, Dawn (In reply to comment #5) > (In reply to comment #3) > > We cannot find the exact cause of failure from the logs... > > > > Do you see that the drivers are correctly installed on the VM? > Yes, the drivers are correctly installed on the VM, I retest again today > with keeping eyes on the guests,find when the NDISTest Client is doing the > last job of "postconfig", the NDISTest Server side finish "postconfig" job > before Client and quickly quit.But the Client is still running "postconfig". > > And another thing is job "InvalidPackets.htm" fail for NDIS6.5, please refer > to the attachment "InvalidPacketsFail.png" > I don't think it is a problem. Postconfig is a local task of removing the protocol driver and reconfiguring the VM if the test did something with it. It is expected for the server to quit. Also the logs are collected from the client side only. > Any way, I'll retest it on new driver 44, and update the result. > > Thanks > Best Regards, > Dawn > > > > Best regards, > > Yan. Please check the event log for traces of the system crash. Thanks, Yan. (In reply to comment #18) > Please check the event log for traces of the system crash. > > Thanks, > Yan. Hi yan, Please refer to the attached "system-event-log.evtx" for logs. Best Regards, Dawn Created attachment 657326 [details]
system-event-log
Hi Dawn,
I see in the event log following errors:
"The server could not bind to the transport \Device\NetBT_Tcpip_{003E256F-3421-468A-BB7B-AC6F5EFAE7DA} because another computer on the network has the same name. The server could not start." (computer name "2k8-32-1-46.virtio.com").
Considering that the DTM uses computer names - this might cause some issues. Can you double check that the VMs used for test and possibly others VMs in the network don't have same name.
Thanks,
Yan.
Hi Dawn,
I see in the event log following errors:
"The server could not bind to the transport \Device\NetBT_Tcpip_{003E256F-3421-468A-BB7B-AC6F5EFAE7DA} because another computer on the network has the same name. The server could not start." (computer name "2k8-32-1-46.virtio.com").
Considering that the DTM uses computer names - this might cause some issues. Can you double check that the VMs used for test and possibly others VMs in the network don't have same name.
Thanks,
Yan.
(In reply to comment #22) > Hi Dawn, > > I see in the event log following errors: > "The server could not bind to the transport > \Device\NetBT_Tcpip_{003E256F-3421-468A-BB7B-AC6F5EFAE7DA} because another > computer on the network has the same name. The server could not start." > (computer name "2k8-32-1-46.virtio.com"). > > > Considering that the DTM uses computer names - this might cause some issues. > Can you double check that the VMs used for test and possibly others VMs in > the network don't have same name. > > Thanks, > Yan. Hi Yan, Yes, actually I saw the logs yesterday, and felt strange since we named the guest as "system name + driver name + build Number + NIC1/2", so shouldn't have the duplicate name,and in the domain if the name duplicate, system will prompt you computer name duplicate. So I still checked two times, didn't find the same name exist for all guest (win2k8-32/64 win2k8-R2). Best Regards, Dawn Since this could be a setup problem (transport binding failure), please try the following:
1. Close all ndis-test related applications running on HCK clients
2. Uninstall NetKVM driver for all RedHat adapters on all clients(Device Manager --> Rings Click --> Uninstall --> Delete driver software)
3. Reboot all clients
4. Install drivers again for all Redhat adapters on all clients
5. Reboot all clients
6. Try to repeat this test
We saw a few similar problems, this sequence usually helps.
(In reply to comment #24) > Since this could be a setup problem (transport binding failure), please try > the following: > 1. Close all ndis-test related applications running on HCK clients We are asked to Run netkvm jobs on WLK ,Could you confirm whether the following steps works on WLK as well ? Mike > 2. Uninstall NetKVM driver for all RedHat adapters on all clients(Device > Manager --> Rings Click --> Uninstall --> Delete driver software) > 3. Reboot all clients > 4. Install drivers again for all Redhat adapters on all clients > 5. Reboot all clients > 6. Try to repeat this test > > We saw a few similar problems, this sequence usually helps. (In reply to comment #25) > (In reply to comment #24) > > Since this could be a setup problem (transport binding failure), please try > > the following: > > 1. Close all ndis-test related applications running on HCK clients > > We are asked to Run netkvm jobs on WLK ,Could you confirm whether the > following steps works on WLK as well ? > Yes, these steps should work on WLK as well. > Mike > > 2. Uninstall NetKVM driver for all RedHat adapters on all clients(Device > > Manager --> Rings Click --> Uninstall --> Delete driver software) > > 3. Reboot all clients > > 4. Install drivers again for all Redhat adapters on all clients > > 5. Reboot all clients > > 6. Try to repeat this test > > > > We saw a few similar problems, this sequence usually helps. Please try the steps in comment #24 (In reply to comment #27) > Please try the steps in comment #24 Hi Yan, Tried with the steps in comment #24 to re-install the netkvm driver, still hit the same issue. Best Regards, Dawn Looks this bug should be filter by https://sysdev.microsoft.com/en-US/Hardware/EC/ECDetails.aspx?id=2966 I will research why filter does not help. Retest it on Build49 on win2k8R2 w/ the driver located in win7 I don't think "InvalidPackages" is the key point cause job failed ,it can be filter by https://sysdev.microsoft.com/en-US/Hardware/EC/ECDetails.aspx?id=2966 from my test The job failed due to "Make Sure communication helpers stats and adapter stats match" sub job Following is the logs : Failed Start Test 12/12/2012 1:06:57.656 PM Make sure communication helper stats and adapter stats match. Error 12/12/2012 1:06:57.656 PM Adapter statistics and comm helper statistics do not match! File: Line: 0 Error Type: WIN32 Error Code: 0x50012 Error Text: There are no more files. End Test 12/12/2012 1:06:57.656 PM Make sure communication helper stats and adapter stats match. Result: Fail Repro: ndistest.exe /logo /auto /client /target:Miniport /tc:{653340A5-E5F0-424E-B98D-C59169DFD541} /TestEnvInfo:TestEnvInfo.xml /support:{F64CE5E3-56C5-4810-BB46-225AC67D6F3E},{653340A5-E5F0-424E-B98D-C59169DFD541} /msg:{ACC8DD1F-8734-4CA9-8352-44218376E597} /jobgroup:WHQL\Lan /ParaVirtualizationDriver:True (In reply to comment #34) > Retest it on Build49 on win2k8R2 w/ the driver located in win7 > I don't think "InvalidPackages" is the key point cause job failed ,it can be > filter by > https://sysdev.microsoft.com/en-US/Hardware/EC/ECDetails.aspx?id=2966 from > my test > > > The job failed due to "Make Sure communication helpers stats and adapter > stats match" sub job > > Following is the logs : > > Failed > Start Test 12/12/2012 1:06:57.656 PM Make sure communication helper stats > and adapter stats match. > Error 12/12/2012 1:06:57.656 PM Adapter statistics and comm helper > statistics do not match! > File: Line: 0 > Error Type: WIN32 > Error Code: 0x50012 > Error Text: There are no more files. > End Test 12/12/2012 1:06:57.656 PM Make sure communication helper stats and > adapter stats match. > Result: Fail > Repro: ndistest.exe /logo /auto /client /target:Miniport > /tc:{653340A5-E5F0-424E-B98D-C59169DFD541} /TestEnvInfo:TestEnvInfo.xml > /support:{F64CE5E3-56C5-4810-BB46-225AC67D6F3E},{653340A5-E5F0-424E-B98D- > C59169DFD541} /msg:{ACC8DD1F-8734-4CA9-8352-44218376E597} /jobgroup:WHQL\Lan > /ParaVirtualizationDriver:True Re-run this job on 2k8R2 5 times ,one time passed . win2k8-32 passed as well ,I will run this job on win2k8-64 Once the win2k8R2 netkvm whql test finished . Best Regards, Mike (In reply to comment #36) > Re-run this job on 2k8R2 5 times ,one time passed . > win2k8-32 passed as well ,I will run this job on win2k8-64 Once the win2k8R2 > netkvm whql test finished . > > Best Regards, > Mike Did not hit this issue on build49 over win7 32/64 ,win2k8 32/64/R2 bit. There must be a configuration issue on QE AD testing environment ,I will re-setup it after all block&netkvm job submit to MSFT. Closing this one Also this erratum will probably help: https://sysdev.microsoft.com/en-US/Hardware/EC/ECDetails.aspx?id=2966 |
Description of problem: Job of "Ethernet - NDISTest6.5" failed with WLK on win2k8/win2k8-R2 Version-Release number of selected component (if applicable): kernel-2.6.32-338.el6.x86_64 qemu-kvm-rhev-0.12.1.2-2.331.el6.x86_64 virtio-win-prewhql-0.1-43 How reproducible: 3/3 Steps to Reproduce: 1. Start guest with CLI: /usr/libexec/qemu-kvm -M rhel6.4.0 -m 6G -smp 4 -cpu cpu64-rhel6,+x2apic -usbdevice tablet -drive file=win2k8-32-nic1.raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,sndbuf=0,id=hostnet0,vhost=on,script=/etc/qemu-ifup-private,downscript=no -device virtio-net-pci,netdev=hostnet0,mac=00:10:19:06:39:01,bus=pci.0,addr=0x4,id=virtio-net-pci0 -netdev tap,sndbuf=0,id=hostnet1,vhost=on,script=/etc/qemu-ifup-private,downscript=no -device virtio-net-pci,netdev=hostnet1,mac=00:10:19:06:39:02,bus=pci.0,addr=0x5,id=virtio-net-pci1 -netdev tap,sndbuf=0,id=hostnet2,script=/etc/qemu-ifup,downscript=no -device e1000,netdev=hostnet2,mac=00:10:19:16:39:03,bus=pci.0,addr=0x6 -uuid 01410ecf-365b-4629-83a6-c7fec786de81 -rtc base=localtime,clock=host,driftfix=slew -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-win2k8-32-nic1,server,nowait -mon chardev=111a,mode=readline -spice port=5931,disable-ticketing -vga qxl -monitor stdio -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 2.Run job of "Ethernet - NDISTest6.5" 3.Run sub job "Start NDISTest Client" Actual results: Job failed without any DTM logs, but only result error info as following: ------------------------------------------------------------------------------- TaskResult Errors # Message Time Source Machine Name Root Cause 1 11/5/2012 5:18:41 PM Execution Agent 2K8-32-43-NIC1 The Execute Task with Commandline cmd /c ndistest.exe /logo /auto /client /target:Miniport /tc:{2DD2A524-E182-47C1-8FA8-2073FF990190} /TestEnvInfo:TestEnvInfo.xml /support:{9CB0A667-F509-4824-9220-9DE440E3447F},{89920EA9-811C-4588-8EE5-C0F11AF7DE43} /msg:{84EDCFFC-EE59-46A3-98C3-AAF567B907C1} /jobgroup:WHQL\Lan /ParaVirtualizationDriver:True Failed with ExitCode 1 Resolution The task exited with an ExitCode other than the ExpectedTaskExitCode. This may cause the Task to Fail if it is set to Fail On Exit Code ------------------------------------------------------------------------------- Expected results: Job should passed without any error. Additional info: No this issue on win2k3-32/64